-- Logs begin at Tue 2022-08-30 03:31:42 UTC, end at Tue 2022-08-30 04:39:35 UTC. -- Aug 30 04:04:40.539557 dv-cinder-852208-11-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Aug 30 04:04:40.548136 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Aug 30 04:04:40.550825 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: *** Starting uWSGI 2.0.18-debian (64bit) on [Tue Aug 30 04:04:40 2022] *** Aug 30 04:04:40.550992 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Aug 30 04:04:40.551136 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Aug 30 04:04:40.551279 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: nodename: dv-cinder-852208-11-iscsi Aug 30 04:04:40.551418 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: machine: x86_64 Aug 30 04:04:40.551560 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: clock source: unix Aug 30 04:04:40.551689 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: pcre jit disabled Aug 30 04:04:40.551817 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: detected number of CPU cores: 4 Aug 30 04:04:40.551956 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: current working directory: / Aug 30 04:04:40.552090 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: detected binary path: /usr/bin/uwsgi-core Aug 30 04:04:40.552229 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: your processes number limit is 47680 Aug 30 04:04:40.552352 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: your memory page size is 4096 bytes Aug 30 04:04:40.552488 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: detected max file descriptor number: 2048 Aug 30 04:04:40.552628 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: lock engine: pthread robust mutexes Aug 30 04:04:40.552919 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: thunder lock: enabled Aug 30 04:04:40.553239 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Aug 30 04:04:40.553450 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Aug 30 04:04:40.593106 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: Python main interpreter initialized at 0x56544ac5d9c0 Aug 30 04:04:40.593106 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: python threads support enabled Aug 30 04:04:40.593106 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: your server socket listen backlog is limited to 100 connections Aug 30 04:04:40.593106 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: your mercy for graceful operations on workers is 90 seconds Aug 30 04:04:40.593478 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: mapped 403077 bytes (393 KB) for 2 cores Aug 30 04:04:40.593548 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: *** Operational MODE: preforking *** Aug 30 04:04:40.594080 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: *** uWSGI is running in multiple interpreter mode *** Aug 30 04:04:40.594080 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: spawned uWSGI master process (pid: 81490) Aug 30 04:04:40.594359 dv-cinder-852208-11-iscsi systemd[1]: Started Devstack devstack@keystone.service. Aug 30 04:04:40.598302 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 1 (pid: 81502, cores: 1) Aug 30 04:04:40.598302 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: spawned uWSGI worker 2 (pid: 81503, cores: 1) Aug 30 04:04:40.598302 dv-cinder-852208-11-iscsi devstack@keystone.service[81490]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Aug 30 04:04:44.993057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 04:04:44.993057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Aug 30 04:04:44.993862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 04:04:44.993862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: from cryptography.utils import int_from_bytes Aug 30 04:04:44.996847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 04:04:44.996847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Aug 30 04:04:44.997307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Aug 30 04:04:44.997307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: from cryptography.utils import int_from_bytes Aug 30 04:04:45.045960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 04:04:45.046163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 04:04:45.046359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] command line args: [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 04:04:45.046634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 04:04:45.046815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ================================================================================ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 04:04:45.047119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.047366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_dir = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.047560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.047749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] config_source = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.047923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] debug = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.048122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.048625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.048714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.049022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.049414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.049628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.049806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_config_append = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.050009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.050218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_dir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.050368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.053375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Aug 30 04:04:45.054744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Aug 30 04:04:45.054744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] command line args: [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Aug 30 04:04:45.054941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.054941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] publish_errors = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Aug 30 04:04:45.055125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ================================================================================ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Aug 30 04:04:45.055263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_dir = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] config_source = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.055925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] debug = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] standard_threads = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] insecure_debug = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_journal = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.056957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_json = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_stderr = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] use_syslog = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_config_append = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.057851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.057935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_dir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.058010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.058086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.058263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.058346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.058520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.058605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.058770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.058859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.059031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.059112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.059204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.059283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.059412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.059490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.059616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.059746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.059860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.059955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.060108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.060192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.060328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.060416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.060563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.060710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.060795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.060869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.061031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.061113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.061281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_format = cadf {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.061377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.061551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.061638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.061803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.062044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.46/identity {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.062146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.062222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] publish_errors = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.062349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.062477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.063081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.063247 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.063335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.063585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.063672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.063973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.064059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.064263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.064357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.064432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.064508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] standard_threads = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.064670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.064751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] strict_password_check = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.064901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.064988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_eventlog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.065262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_journal = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.065465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_json = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_stderr = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.065764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] use_syslog = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.065834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] watch_log_file = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Aug 30 04:04:45.066090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.066923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.067938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.068955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.069901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.070864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.071967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.072994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.073966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.074977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.075983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.076890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.077892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.078010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.078087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.078216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.080137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.080822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.080916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.081657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.082960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.083168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.083243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.083316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.083397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.083479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.084972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.085937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.external = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.enabled = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.password = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.086943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] auth.token = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_password = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.087993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_sasl_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.memcache_username = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.088929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.089991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.090928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.091904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.external = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.mapped = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.092845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.password = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] auth.token = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.093963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.094908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.095990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.096923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.097919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.098904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.099994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.100083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.100965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.101935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.102992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.104975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.105848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.106884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.107851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.108868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.109782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.110050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.110138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.110483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.110567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.111979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.password = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.112969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.113963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.114868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.115988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.116973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117247 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117247 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.117922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.118914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.119909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.120881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.121919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.122926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.123898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.124984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.125628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.125767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.125904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.126980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.127897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.128871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.129933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] role.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.130007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.130081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.130154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.130228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.130660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.131936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.132957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.133973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.134944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.135989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.provider = fernet {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.136984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.137139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.137230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.137294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81502) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 04:04:45.137383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.137629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.137878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.138117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.138382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.138621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.138870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.139148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.139382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.139628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.139898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.140163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.140410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.140659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.141523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.141967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.142366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.142617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.142857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.143099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.143351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.143593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.143843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.144082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.144335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.144576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.144822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.145054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.145295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.145530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Aug 30 04:04:45.145701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=81503) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Aug 30 04:04:45.167753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.167965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.168189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.171777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.172191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.172524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.176744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.181076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.205570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.205800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.207880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.208110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.259147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.259524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.308392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.308621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.309924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.310211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.465582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.465798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.466010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.466228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.466474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.466690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.483299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.483526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.483747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.483964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.484178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.484388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.490854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.491090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.491317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.491540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.491758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.491973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.492188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.497287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.497518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.497741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.497956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.498167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.498400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.498609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.503119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.506646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.516265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.516495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.516719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.517004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.517257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.517519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.517665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81502) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.518170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.518431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.518656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.518875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.519118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.519331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.519542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=81503) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Aug 30 04:04:45.535475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.535594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.535677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 04:04:45.535754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 04:04:45.536010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 04:04:45.536291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Aug 30 04:04:45.536474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.536997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.537247 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.537332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.537509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.546994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.547274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.547525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.547672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.547801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.547903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.548071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.548352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.554772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.555307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `type` (API Prefix: ) [/credentials, /credentials/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.557656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.557942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.558228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.558469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `type` (API Prefix: ) [/domains, /domains/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.558555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.558800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.558898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.559961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.560148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.560241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.560416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.560501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.560797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.561077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.561345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.583781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.584999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.585252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.585535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.586491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `type` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.586774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.590066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.590371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.590655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.591548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `type` (API Prefix: ) [/groups, /groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.591834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.592169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.597666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.597947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.599330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `type` (API Prefix: ) [/limits, /limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.599607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.602072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.602391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.602677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.602961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.603255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.604021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `type` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.604311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.604597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.604999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.605980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.606272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.621377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.621656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.623957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.624263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.625522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.628541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `type` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.628653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.628890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.631772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.632069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.633165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.636264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.636647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `type` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.639329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.639630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.639774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `type` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.639915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.640201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.640482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.641640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.643708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.644018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.644304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.644601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.644878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.645157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.658081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.658393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.658669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.658935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.662239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `type` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.662549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.662814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.663075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.665649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.667326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.668871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.669148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.669868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH2: ['/token' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.671418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.671810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.672136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.672486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.672788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.673078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.673162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.673400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.676075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `type` (API Prefix: ) [/policies, /policies/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.676378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.676765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.677052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.677344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.684391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.684676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.684959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.685242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.685527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.685807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.686105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.689749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `type` (API Prefix: ) [/projects, /projects/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.690043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.690454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.690744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.691033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.691311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.691603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.702229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.705990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.707595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `type` (API Prefix: ) [/regions, /regions/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.708904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.710383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.710515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `type` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.711863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.712147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.712432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.713406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.714883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.716556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `type` (API Prefix: ) [/roles, /roles/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.716798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.717095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.719147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.720631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.723513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.723824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.724114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.724404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.732340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.732630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.732912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.737322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.738853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `type` (API Prefix: ) [/services, /services/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.740079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81502) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.740366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.740688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.740975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.741248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.741527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.741802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.741990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.742996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.743161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.743427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.743709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81502) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.751710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `type` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.751998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.752302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.759357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `type` (API Prefix: ) [/users, /users/] {{(pid=81503) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Aug 30 04:04:45.759651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.759942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.760228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.760505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.760782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.762989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=81503) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Aug 30 04:04:45.779580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56544ac5d9c0 pid: 81502 (default app) Aug 30 04:04:45.783992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706645-0a68-46ee-8ae1-d6e6c8604eaf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:45.784278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706645-0a68-46ee-8ae1-d6e6c8604eaf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:45.784550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9706645-0a68-46ee-8ae1-d6e6c8604eaf None None] PATH_INFO: `/v3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:45.788727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1/1] 10.222.0.46 () {50 vars in 791 bytes} [Tue Aug 30 04:04:45 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:04:45.799847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WSGI app 0 (mountpoint='') ready in 5 seconds on interpreter 0x56544ac5d9c0 pid: 81503 (default app) Aug 30 04:04:52.501864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:52.502412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:52.502412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:52.503732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 2/2] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:04:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:04:52.511061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1181-583e-40a3-a972-47858efda19e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:52.511517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1181-583e-40a3-a972-47858efda19e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:52.511659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebe1181-583e-40a3-a972-47858efda19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:52.582748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: INFO dbcounter [None req-cebe1181-583e-40a3-a972-47858efda19e None None] Registered counter for database keystone Aug 30 04:04:52.601608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG oslo_db.sqlalchemy.engines [None req-cebe1181-583e-40a3-a972-47858efda19e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81503) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 04:04:52.607932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writer thread running {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 04:04:52.791990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-cebe1181-583e-40a3-a972-47858efda19e None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81503) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 04:04:52.792291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG passlib.handlers.bcrypt [None req-cebe1181-583e-40a3-a972-47858efda19e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81503) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 04:04:52.851194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cebe1181-583e-40a3-a972-47858efda19e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:04:52.852804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cebe1181-583e-40a3-a972-47858efda19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:04:52.987420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1/3] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:04:52 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 479 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 04:04:53.476938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:53.547729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: INFO dbcounter [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] Registered counter for database keystone Aug 30 04:04:53.569460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG oslo_db.sqlalchemy.engines [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=81502) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Aug 30 04:04:53.575681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writer thread running {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Aug 30 04:04:53.694557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d006d294-e037-4ba6-88ec-7d0724868f03 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:53.698866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:53.699496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:53.699779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:53.700048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:53.708490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:53.761229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING keystone.common.rbac_enforcer.enforcer [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 04:04:53.761515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603d6ed8-5044-422a-b655-55ee7b31d194 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:53.771872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 3/4] 10.222.0.46 () {58 vars in 1324 bytes} [Tue Aug 30 04:04:53 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 04:04:53.778233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe1181-583e-40a3-a972-47858efda19e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:53.793639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebe1181-583e-40a3-a972-47858efda19e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:53.798448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:53.799548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:53.799833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:53.800108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] PATH_INFO: `/v3/projects/b409aaa1029446de8dc3bfb718ab67c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:53.801705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] RBAC: Authorizing `identity:get_project(project_id=b409aaa1029446de8dc3bfb718ab67c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:53.856953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING keystone.common.rbac_enforcer.enforcer [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Aug 30 04:04:53.857303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d338787-86c3-4ab6-b99c-a29dd294d776 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:53.862415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 2/5] 10.222.0.46 () {56 vars in 1207 bytes} [Tue Aug 30 04:04:53 2022] GET /identity/v3/projects/b409aaa1029446de8dc3bfb718ab67c6 => generated 344 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:04:55.759447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562296-4b52-4899-851a-9d3909b666a1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:55.759977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562296-4b52-4899-851a-9d3909b666a1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:55.759977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47562296-4b52-4899-851a-9d3909b666a1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:55.764060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 4/6] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:04:55 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:04:55.769580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:55.769995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:55.770123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:55.790528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:04:55.792091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:04:55.801648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 3/7] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:04:55 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Aug 30 04:04:56.067046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47562296-4b52-4899-851a-9d3909b666a1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:56.080744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47562296-4b52-4899-851a-9d3909b666a1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:56.084956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:56.085553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:56.085833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:56.086103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:56.093771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:56.096106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c38120-c43a-40dc-a051-c82034e36f91 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:56.104762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 5/8] 10.222.0.46 () {58 vars in 1324 bytes} [Tue Aug 30 04:04:56 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:04:56.110805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:56.120445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05af5e39-d5b2-4599-ade6-ed8acfb34796 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:56.124946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:56.125714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:56.125985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:56.126269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:56.127597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:56.129425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeeb819e-7d61-447a-ad65-59d3607cccf2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:56.131004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 4/9] 10.222.0.46 () {56 vars in 1198 bytes} [Tue Aug 30 04:04:56 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:04:58.102929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2f7d-cfdf-434a-8753-e94cef7ad93a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.103491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2f7d-cfdf-434a-8753-e94cef7ad93a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.103491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fd2f7d-cfdf-434a-8753-e94cef7ad93a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.104800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 6/10] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:04:58.110511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.111020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.111230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.129712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:04:58.130913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:04:58.157879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d405-3529-4e8f-8f2e-e76d1400701a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.158153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d405-3529-4e8f-8f2e-e76d1400701a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.158454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e02d405-3529-4e8f-8f2e-e76d1400701a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.159777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 7/11] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:04:58.165231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.165511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.165780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.208746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] detected 'bcrypt' backend, version '4.0.0' {{(pid=81502) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Aug 30 04:04:58.209140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG passlib.handlers.bcrypt [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=81502) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Aug 30 04:04:58.213839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 5/12] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:04:58.259047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:04:58.260627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:04:58.361110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 8/13] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:04:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:04:58.481842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.507633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:58.591114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e9b5cb-ee86-4527-8e07-782ce74b7d67 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:58.624831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:58.625489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.625767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.626039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.636063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.637026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:04:58.646322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:04:58.647437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:04:58.648300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 6/14] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:04:58.653324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.737362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:58.743417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96b54a0-dfb9-4055-8ac0-82c47cbc0fbd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:58.745683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:58.756377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b865b1c0-e419-4e86-ba7a-ae503046a9d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:58.782077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:58.782968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.783251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.783520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.790768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:58.791372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.791649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.791918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.792590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:04:58.792888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:58.794305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:58.806871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 7/15] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:04:58.812474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.856311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:58.857839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:58.915292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1a0ecb-bfb7-41c3-a716-eb036180ff1f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:58.951189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:58.951820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:58.952097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:58.952367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:58.954277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:58.955759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:58.956789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 9/16] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 04:04:58.957350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 8/17] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:04:58.962372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.962605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:58.965152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:58.977893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:59.061676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c453edfb-3869-40c4-8650-4c0cef3e721d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:59.062135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7407a1-784b-469b-973b-339862371c0e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:59.095656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:59.096279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:59.096499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:59.096628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:59.096807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:59.097348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:59.097624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:59.097892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:59.097987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:59.099364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:59.101057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 9/18] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:04:59.106688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:59.120592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:59.122092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:04:59.147340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:04:59.148339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:04:59.149155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:04:59.149155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:04:59.149155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:04:59.149155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 10/19] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:04:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:04:59.155036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:04:59.201480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:59.213851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:04:59.222740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a67ecf-1d05-4c08-a2b3-a1e4f0f81e89 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:59.257169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:59.257787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:59.258073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:59.258379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:59.259112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.domain.id=default, user.id=01e05990467f46b9b8ba90e062973118) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:04:59.259417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:59.260889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6344d243-6f02-481a-b6e0-35c932f00635 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:59.263181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de46bbb9-acc9-4815-9b36-c1d503c03d9a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:04:59.267880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 10/20] 10.222.0.46 () {56 vars in 1322 bytes} [Tue Aug 30 04:04:59 2022] GET /identity/v3/role_assignments?user.id=01e05990467f46b9b8ba90e062973118&scope.domain.id=default&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:04:59.299867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:04:59.300495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:04:59.300770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:04:59.301038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:04:59.301701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:04:59.302000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:04:59.303476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e586cb9a-a3cc-4d4e-923c-83e9880bbc8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:04:59.323585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 11/21] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:04:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:01.278322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f4d9-37d1-4288-abcc-af8c344e5acc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.278844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f4d9-37d1-4288-abcc-af8c344e5acc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.279137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4f4d9-37d1-4288-abcc-af8c344e5acc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.282131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 11/22] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:01.291942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3eab-5a92-4bbd-9719-28459622d91d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.292494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3eab-5a92-4bbd-9719-28459622d91d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.292494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6be3eab-5a92-4bbd-9719-28459622d91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.319928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f6be3eab-5a92-4bbd-9719-28459622d91d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:01.321171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f6be3eab-5a92-4bbd-9719-28459622d91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:01.421286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 12/23] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:01.703490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1a46-bd4e-4328-ae91-c8db9c7f0cbb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.703892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1a46-bd4e-4328-ae91-c8db9c7f0cbb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.704303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1f1a46-bd4e-4328-ae91-c8db9c7f0cbb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.706672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 12/24] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:01.711652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583467-4f2c-49af-9b38-ccd9ee1b1320 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.712167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583467-4f2c-49af-9b38-ccd9ee1b1320 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.712659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b583467-4f2c-49af-9b38-ccd9ee1b1320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.739308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0b583467-4f2c-49af-9b38-ccd9ee1b1320 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:01.740851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0b583467-4f2c-49af-9b38-ccd9ee1b1320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:01.792444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:01.828836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 13/25] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:01.922917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd2c1-80a4-4e79-bf59-4697a78d4920 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.923556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd2c1-80a4-4e79-bf59-4697a78d4920 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.924025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7dd2c1-80a4-4e79-bf59-4697a78d4920 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.925807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 13/26] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:01.931923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:01.932212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:01.932491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:01.959366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:01.959815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd2c1-80a4-4e79-bf59-4697a78d4920 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:01.960489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:02.078019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 14/27] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:02.174960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7dd2c1-80a4-4e79-bf59-4697a78d4920 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:02.219335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:02.219867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.220147 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.220415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.239404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:02.244372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:02.245229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:02.246440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 14/28] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 287 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:02.252573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:02.269709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:02.275659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:02.443538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c27a4e6-a772-44d9-a54c-87f64890d4bb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:02.464861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:02.492866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:02.493419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.493702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.493972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.494674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:02.494976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:02.496379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:02.524131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 15/29] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:02.525252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:02.541134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4b7deb-c07c-45b8-a8dd-7a0e8d5f859e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:02.601235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:02.602134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.602492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.602772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.603682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:02.605758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632dd76a-1e4e-4485-a6c0-b4d8a7a60ad9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:02.664615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ad68387-0727-4bcb-93da-d70162ce7d38 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:02.703206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:02.704333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.704809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.705226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.716415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:02.718282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:05:02.737311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:02.738550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:05:02.739559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:05:02.739559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:05:02.739559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:02.741850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 16/30] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:02.742988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:02.757897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 15/31] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 30 04:05:02 2022] POST /identity/v3/roles => generated 219 bytes in 489 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:05:02.759917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26541c-e5e1-4bb3-b27e-58b8cfc5bb7e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.760473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26541c-e5e1-4bb3-b27e-58b8cfc5bb7e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.760908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26541c-e5e1-4bb3-b27e-58b8cfc5bb7e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.762410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 16/32] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:02.767988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26541c-e5e1-4bb3-b27e-58b8cfc5bb7e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:02.776789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:02.874671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b358f4a-abc6-4006-acec-187b94181a25 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:02.883756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:02.918125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:02.918803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.919081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.919390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.920912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:02.922669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd959763-77d6-4a86-b558-dc99ad418702 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:02.924968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 17/33] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:02 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:02.925961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:02.926243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:02.930275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:02.947133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:02.948760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:02.966359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f26541c-e5e1-4bb3-b27e-58b8cfc5bb7e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.031893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 18/34] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:03.032822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:03.039393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.040372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.040873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.041390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.042415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:03.042970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.044959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.053439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:03.077635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 17/35] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:05:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:03.083678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:03.149699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310269e0-9739-4865-b327-e33fbaed08c3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.191491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.192275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.192560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.192941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.194083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.195557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e34887c-9844-4c61-a170-59dd18f75df6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.197874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 19/36] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:03.206288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ade37-8fe0-4230-b481-e96e60125e73 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.206288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ade37-8fe0-4230-b481-e96e60125e73 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.206288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ade37-8fe0-4230-b481-e96e60125e73 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.206608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 20/37] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:03.211042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ade37-8fe0-4230-b481-e96e60125e73 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:03.244330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3c928a-9f42-46d0-b745-2feb82f7308b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.278664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:03.291170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.292177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.292676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.293903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.294794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.296428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42794b7-7f23-4332-bd54-8edee0c3fad3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.316573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:03.332035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ade37-8fe0-4230-b481-e96e60125e73 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.342509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 18/38] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:05:03 2022] POST /identity/v3/projects => generated 301 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:05:03.344351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad18f2-56f9-4a13-96a1-8896ce209fd6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.344731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad18f2-56f9-4a13-96a1-8896ce209fd6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.345049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ad18f2-56f9-4a13-96a1-8896ce209fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.367391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-43ad18f2-56f9-4a13-96a1-8896ce209fd6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:03.369395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-43ad18f2-56f9-4a13-96a1-8896ce209fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:03.394900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.395840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.396217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.396590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] PATH_INFO: `/v3/domains/default/users/01e05990467f46b9b8ba90e062973118/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.405408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=01e05990467f46b9b8ba90e062973118, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.408581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.464126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 19/39] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:03.465241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54743786-f94e-4fed-b414-e251107f6a94 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.465528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54743786-f94e-4fed-b414-e251107f6a94 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.465798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54743786-f94e-4fed-b414-e251107f6a94 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.467055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 20/40] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:03.473244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.473656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.474013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.556691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 21/41] 10.222.0.46 () {58 vars in 1361 bytes} [Tue Aug 30 04:05:03 2022] PUT /identity/v3/domains/default/users/01e05990467f46b9b8ba90e062973118/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 349 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:03.557901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:03.622526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:03.624512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:03.702596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce472e9f-54f9-4de9-b046-05f5cea00c06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.719916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 21/42] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:03.751684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.752313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.752595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.752865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.753550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.755186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.780171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:03.788344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 22/43] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 30 04:05:03 2022] POST /identity/v3/roles => generated 225 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:03.835402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:03.913305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fded1893-0d77-4865-9c02-825a041ca7e4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:03.953259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:03.954010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:03.954391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:03.954671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:03.955357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:03.956729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:03.977503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 22/44] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 30 04:05:03 2022] POST /identity/v3/roles => generated 223 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:05:04.041554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:04.052262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:04.155654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b4c28f-4a23-4bf9-a3f0-c9045bcf9f43 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:04.196383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:04.197024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:04.197305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:04.197577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:04.198839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:04.201272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd9874e-bab7-441d-b83b-00005691ebc9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:04.203392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 23/45] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:04.212749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:04.310359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:04.317025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f021d46-d0ff-4af8-819b-d1603ac674dc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:04.352236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:04.352849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:04.353131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:04.353408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:04.354067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:04.355556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ed05c3-961b-4697-bdac-19cc56b01f29 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:04.380892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 23/46] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:05:04 2022] POST /identity/v3/projects => generated 312 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:05.622362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301746-4c35-4ffb-abbc-7302e28f76b5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:05.622909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301746-4c35-4ffb-abbc-7302e28f76b5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:05.622909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1301746-4c35-4ffb-abbc-7302e28f76b5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:05.624300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 24/47] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:05.629898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:05.630325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:05.630564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:05.649571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:05.650780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:05.715790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 24/48] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:05.981573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301746-4c35-4ffb-abbc-7302e28f76b5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.058167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:06.085328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1301746-4c35-4ffb-abbc-7302e28f76b5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.120296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.120930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.121216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.121492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.134432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:06.141239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:06.142519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:06.143270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 25/49] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:06.148149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.225950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:06.248239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e5d25-ca2d-4ae3-b2bb-3e62bfc3d1b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.283799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.284413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.284700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.284972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.285638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:06.285943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:06.287381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e526d93-d988-4cd6-9d4a-c96bac7335cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:06.295512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 25/50] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:06.301600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.337807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d279ef-3440-407e-b53b-bf09400c9da1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.338100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d279ef-3440-407e-b53b-bf09400c9da1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.338389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d279ef-3440-407e-b53b-bf09400c9da1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.339732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 26/51] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:06.345213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.345498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.345775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.364029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:06.365226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:06.402926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d995f5-650f-485d-8c6c-bbff38b1f2cb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.433978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 27/52] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:06.436717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.437590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.437869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.438192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.439365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:06.440804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:06.442504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 26/53] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:06 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:06.447931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.520842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:06.552111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1259ce3-42f1-49c0-8588-0021b0a0248c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.586261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.586919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.587199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.587471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.588590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:06.590042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:06.591640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 28/54] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:06.599848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.602963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:06.734731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.807673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00415bbe-b32e-4154-bd1b-475eae85b5a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.842729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26acd69-b361-4e9d-b5cd-548a794fda3d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:06.843996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.844648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.844922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.845188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.845913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.domain.id=default, user.id=01e05990467f46b9b8ba90e062973118) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:06.846217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:06.847677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:06.855570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 27/55] 10.222.0.46 () {56 vars in 1322 bytes} [Tue Aug 30 04:05:06 2022] GET /identity/v3/role_assignments?user.id=01e05990467f46b9b8ba90e062973118&scope.domain.id=default&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 530 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:06.878984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:06.879646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:06.879926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:06.880201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:06.881388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:06.882958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c079e196-24b2-40dd-9ec9-16f96593b91a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:06.884939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 29/56] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:06.891740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:06.963405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:07.001921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e205b3-1faf-4f26-9ad7-158455db24f5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:07.037847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:07.038507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:07.038792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:07.039067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:07.039757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:07.041198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1dba490-18ae-46b8-a738-80b0e9219fd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:07.071053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 28/57] 10.222.0.46 () {60 vars in 1140 bytes} [Tue Aug 30 04:05:06 2022] POST /identity/v3/projects => generated 298 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:08.972126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bc23-2567-4db9-846e-5da71753ffe3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:08.972635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bc23-2567-4db9-846e-5da71753ffe3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:08.972818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed81bc23-2567-4db9-846e-5da71753ffe3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:08.974353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 30/58] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:08.979340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:08.979719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:08.979898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:08.998786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:09.000480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:09.065307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 29/59] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:09.338355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bc23-2567-4db9-846e-5da71753ffe3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:09.347824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:09.441555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed81bc23-2567-4db9-846e-5da71753ffe3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:09.475729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:09.476351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:09.476632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:09.476908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:09.478016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:09.479500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04959c1-a69b-4bb7-87f5-8ae16f0cc6ca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:09.481058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 31/60] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:09 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:09.486964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:09.537626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:09.589342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0295d99b-df88-41b7-82ab-73701225f436 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:09.624088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:09.625482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:09.625631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:09.626616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:09.627900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:09.629894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c5882-df2a-4c11-b80e-b0b79f017052 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:09.657825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 30/61] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:09 2022] POST /identity/v3/users => generated 275 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:12.732688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27463-0310-4d94-b318-33e849773ea6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:12.734703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27463-0310-4d94-b318-33e849773ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:12.735306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a27463-0310-4d94-b318-33e849773ea6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:12.737256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 32/62] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:12.742230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb996b-1344-4d48-b8ae-22d40c2a05ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:12.743071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb996b-1344-4d48-b8ae-22d40c2a05ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:12.743608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb996b-1344-4d48-b8ae-22d40c2a05ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:12.771318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4abb996b-1344-4d48-b8ae-22d40c2a05ac None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:12.772870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4abb996b-1344-4d48-b8ae-22d40c2a05ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:12.876640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 31/63] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:12.890510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44681f-fe1d-4aea-a15d-bea9e94446a0 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:12.890837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44681f-fe1d-4aea-a15d-bea9e94446a0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:12.891170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a44681f-fe1d-4aea-a15d-bea9e94446a0 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:12.892518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 33/64] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:12.897637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc4dc4-3f5c-4a71-9153-d4cc3a93809a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:12.897935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc4dc4-3f5c-4a71-9153-d4cc3a93809a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:12.898210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55bc4dc4-3f5c-4a71-9153-d4cc3a93809a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:12.925556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-55bc4dc4-3f5c-4a71-9153-d4cc3a93809a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:12.926809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-55bc4dc4-3f5c-4a71-9153-d4cc3a93809a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:12.967608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.021865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 32/65] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:13.029584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fa86-f9c6-4c16-810c-dbb73fedabde None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.030128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fa86-f9c6-4c16-810c-dbb73fedabde None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.031243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d65fa86-f9c6-4c16-810c-dbb73fedabde None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.032827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 34/66] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:13.038284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.038581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.038859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.063285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:13.064495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:13.150039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 33/67] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:13.203498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65fa86-f9c6-4c16-810c-dbb73fedabde None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.313319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.351960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d65fa86-f9c6-4c16-810c-dbb73fedabde None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:13.402990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:13.403651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.403935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.404211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.419966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:13.425803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:13.427773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:13.428528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 35/68] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:13.436619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.454605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.579955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef45d08-8c29-4b19-ae51-95a4731211ae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:13.594019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.605244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb4fc90-f62b-430a-ac7a-3e78887c4320 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:13.623423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:13.626057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.626057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.626057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.627940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.641961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-09780416-c659-49b6-84cf-2003c6ad5f1c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:13.647619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:13.648600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:13.649633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 36/69] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:13.652429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a9c8-ccf2-4b41-8764-f95973d1bfd2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.652429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a9c8-ccf2-4b41-8764-f95973d1bfd2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.652429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d70a9c8-ccf2-4b41-8764-f95973d1bfd2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.652622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 37/70] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:13.653310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70a9c8-ccf2-4b41-8764-f95973d1bfd2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.654439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:13.655125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.655536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.657003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:13.657238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:13.658974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dccee6d-98a6-4947-9871-f5dae9cd77e7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:13.669699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 34/71] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:13.671005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52862-ba72-4740-a92f-34309f85e3db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.671315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52862-ba72-4740-a92f-34309f85e3db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.671587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa52862-ba72-4740-a92f-34309f85e3db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.672944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 35/72] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:13.674375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52862-ba72-4740-a92f-34309f85e3db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.799287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d70a9c8-ccf2-4b41-8764-f95973d1bfd2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:13.806996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa52862-ba72-4740-a92f-34309f85e3db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:13.841114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:13.841798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.842074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.842372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.851545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:13.852545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.853002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.853818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.854563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:13.854870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:13.856837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:13.857025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3f075477-169b-4c0a-8671-c2461c80b90f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:13.864081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:13.865183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:13.866030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 38/73] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:13.866721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0b38b-1caf-465e-970b-524a5ef14681 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.867014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0b38b-1caf-465e-970b-524a5ef14681 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.867289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b0b38b-1caf-465e-970b-524a5ef14681 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:13.873610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 36/74] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:13.873610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:13.889413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-97b0b38b-1caf-465e-970b-524a5ef14681 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:13.891421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97b0b38b-1caf-465e-970b-524a5ef14681 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:13.896386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.954803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:13.976834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 39/75] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:13.978569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:13.978860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:13.979205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.001780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825bf757-329f-4e19-be4d-3b7b5d8aa50b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.008048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:14.009372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:14.051184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.051838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.052127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.052412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.071002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.072493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.074307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 37/76] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:13 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.075655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.100950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 40/77] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:14.102163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.188700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.219541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2f1575-616b-46c5-99fd-d14a961816b1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.229287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d6b75f-70da-4dda-b08d-86daa11407e3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.245369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.260866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.261616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.261907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.262184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.263051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:14.263415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.265063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.267883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.268518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.268795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.269068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.270285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.271808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.273576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 41/78] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.274443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.275399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 38/79] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.281751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.388965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8d096-9f43-4c85-b6f2-5ef70873761c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.396493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0828e1e4-a4e0-4dc6-8b93-748f0fd87b8b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.426827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.427443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.427728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.428003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.433394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.434008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.434351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.434667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.444459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.481336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.502320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.503824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.506575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.508037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.607948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 39/80] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:14.607948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.618727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 42/81] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:14.618727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.650235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.703455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.715808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c500b7-ce91-426d-8849-79e95ed7fb84 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.721972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56723db-745f-48b7-b97c-efeb68092187 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.753644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.754499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.754783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.755115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.756457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.757212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.757839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.758122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.758207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.758431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.765440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 40/82] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.765440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.770366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:14.775540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:14.776522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 43/83] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/roles/member => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:14.782988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.874083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50caeb9e-58f4-47e7-be82-0e8ba5187e66 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.882857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:14.892579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424fc12-74cf-42b7-ab5b-e2361dd1890a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:14.915460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.916015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.916294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.916562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.917664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.919270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.921022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 41/84] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.921648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.931789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:14.932520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:14.932810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:14.933088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:14.934225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:14.935739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:14.937974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 44/85] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:14.938982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:14.951525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.026072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5760728d-36f2-4a20-ae55-f9eaf3702f23 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.041887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2b363e-3b89-4acb-aef0-f7587b7b297f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.060657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.061291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.062140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.062527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.063721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.065161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.066836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 42/86] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.067690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.076544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.077158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.078174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.078901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.116244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.147687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.149498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.162068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.174039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbea35c6-6c1e-4531-b2c0-faf1c6c5d360 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.207958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.208561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.208840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.209111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.209769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:15.210069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.211476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.219762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 43/87] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.220529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.290319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 45/88] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:15.295349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.336093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75c201e-b59a-444c-a6ab-2066ded33fc1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.338523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.372457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.373196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.373481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.373762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.375212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.377038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.395794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97316c24-58c6-48d4-8c08-0c2493d94db0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.399877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 44/89] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:15 2022] POST /identity/v3/users => generated 289 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:15.410477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.410568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.440577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.441143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.441424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.441695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.442469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=01e05990467f46b9b8ba90e062973118) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:15.442781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.444243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.450364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 46/90] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/role_assignments?user.id=01e05990467f46b9b8ba90e062973118&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.451257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.552445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a024ba-258c-48b3-b494-52f94c15eac2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.601572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a21a2-5935-4ac2-95ad-594c79f2f0a5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.606383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.607284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.607705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.608124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.609233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:15.609943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.612310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.619680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 45/91] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 259 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.620881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.656605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.657930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.658203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.658517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.659799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.661286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.663014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 47/92] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.666759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.669044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.725903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:15.784872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b28100-3a33-4848-b7a3-1be7b5996a8a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.810094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c448c1d4-0b8c-4eca-8887-f58fbf6084f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:15.840888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.841707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.842078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.842482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:15.845252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:15.846885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:15.848737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 46/93] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:15.863671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:15.877309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:15.878362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:15.878657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:15.878994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:16.010013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:16.011364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:16.015545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:16.028521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175c0fe0-b666-47ae-b86f-fe89b53b589e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:16.066722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:16.107049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:16.107691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:16.107977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:16.108254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:16.109263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:16.109769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:16.111287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:16.117379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 47/94] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=793acbb060a2436192eb2c7413739e83 => generated 259 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:16.134858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 48/95] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:15 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 04:05:16.142039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:16.287850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faff4590-e03a-4c5b-bb68-22edb8e5b25e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:16.334609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:16.335256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:16.335540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:16.335811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] PATH_INFO: `/v3/projects/55852f8e0358419196fa1f5337bdc887` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:16.336956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] RBAC: Authorizing `identity:get_project(project_id=55852f8e0358419196fa1f5337bdc887)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:16.338447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a3e5d40-fbfc-4c5f-84ee-201ac551347a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:16.340222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 48/96] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:16 2022] GET /identity/v3/projects/55852f8e0358419196fa1f5337bdc887 => generated 312 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:16.346583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:16.476748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0fa6b2-1edf-4ab2-bfb5-ae74d8b65714 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:16.518191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:16.519162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:16.519657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:16.520117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:16.521037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=55852f8e0358419196fa1f5337bdc887, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:16.521573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:16.523314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0fd75d2-744f-43ec-b428-f16a202fb668 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:16.534367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 49/97] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:16 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=55852f8e0358419196fa1f5337bdc887&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 259 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:18.134091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf085-52c4-4232-abf7-770c569c92f0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.134676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf085-52c4-4232-abf7-770c569c92f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.134676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffaf085-52c4-4232-abf7-770c569c92f0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.136045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 49/98] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:18.145162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b46f-9c7c-4f68-8495-46fb77fabbd4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.145609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b46f-9c7c-4f68-8495-46fb77fabbd4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.145792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f51b46f-9c7c-4f68-8495-46fb77fabbd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.164448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:18.175676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3f51b46f-9c7c-4f68-8495-46fb77fabbd4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:18.177059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3f51b46f-9c7c-4f68-8495-46fb77fabbd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:18.276812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 50/99] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:18.580595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cc6f0-70bb-4e11-86d4-5f886a0831fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.580938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cc6f0-70bb-4e11-86d4-5f886a0831fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.581051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cc6f0-70bb-4e11-86d4-5f886a0831fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.582554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 50/100] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:18.587869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3833f2-1c14-4cdf-bd50-8ea55fc5a809 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.588395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3833f2-1c14-4cdf-bd50-8ea55fc5a809 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.588896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3833f2-1c14-4cdf-bd50-8ea55fc5a809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.611190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7e3833f2-1c14-4cdf-bd50-8ea55fc5a809 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:18.612718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e3833f2-1c14-4cdf-bd50-8ea55fc5a809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:18.713225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 51/101] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:18.728974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38905-9860-4f56-8f91-cc14b6e39ea4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.729577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38905-9860-4f56-8f91-cc14b6e39ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.730058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2e38905-9860-4f56-8f91-cc14b6e39ea4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.731647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 51/102] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:18.737592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bae12-2813-4e57-b35c-507ff1886763 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.737881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bae12-2813-4e57-b35c-507ff1886763 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.738158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509bae12-2813-4e57-b35c-507ff1886763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.763236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-509bae12-2813-4e57-b35c-507ff1886763 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:18.764435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-509bae12-2813-4e57-b35c-507ff1886763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:18.851292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e38905-9860-4f56-8f91-cc14b6e39ea4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:18.862620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:18.876137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 52/103] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:18.892741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:18.903079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1f6e-60e2-415a-86cf-77bef780123c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.903360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1f6e-60e2-415a-86cf-77bef780123c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.903638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee1f6e-60e2-415a-86cf-77bef780123c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.905862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 53/104] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:18.911682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:18.912178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:18.912849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:18.932332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:18.933887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:19.031302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2e38905-9860-4f56-8f91-cc14b6e39ea4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.082636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.083292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.083585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.083892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.096358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 54/105] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:19.099923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:19.112386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:19.113473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:19.114453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:19.116404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 52/106] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:19.122346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.151242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.254909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.278138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c38eb7-7caa-4046-9105-a9b13101250f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.290299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71fbbac8-d506-476c-999d-1a9ad8fca221 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.317178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.318075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.318535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.318939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.319417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.321476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:19.324100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48d8b71c-9ad2-4e00-a57f-1595b2ab24a3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:19.326430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 53/107] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:19.328784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129086b-a3e5-4106-a902-69ed75cb9635 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.329235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129086b-a3e5-4106-a902-69ed75cb9635 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.329685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4129086b-a3e5-4106-a902-69ed75cb9635 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.331114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 54/108] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:19.335785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129086b-a3e5-4106-a902-69ed75cb9635 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.335911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317050f8-8275-4594-926e-0289f6625adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.335911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317050f8-8275-4594-926e-0289f6625adf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.335911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317050f8-8275-4594-926e-0289f6625adf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.335911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317050f8-8275-4594-926e-0289f6625adf None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.336277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317050f8-8275-4594-926e-0289f6625adf None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:19.336277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317050f8-8275-4594-926e-0289f6625adf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:19.337655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317050f8-8275-4594-926e-0289f6625adf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:19.347479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 55/109] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:19.348196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317050f8-8275-4594-926e-0289f6625adf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.451481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4129086b-a3e5-4106-a902-69ed75cb9635 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.476667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317050f8-8275-4594-926e-0289f6625adf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.487385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.488028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.488314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.488611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.499927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-39cc4baa-5be9-4b18-ac8b-bfa79e117323 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:19.504829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:19.505909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:19.506998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 55/110] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:19.507818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.507818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.507818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.526697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.527320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.527604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.527879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.528538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:19.530098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:19.533942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:19.535173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:19.544013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.558402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 56/111] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:05:19 2022] POST /identity/v3/projects => generated 302 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:19.558402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.612366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 56/112] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:19.619287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.623616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.728055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16ba030-7474-4d26-98f5-eeb4a8ca214a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.761449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1926ab33-e64e-4c1e-bb28-1168a104f1a9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.770337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.770988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.771273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.772190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.773384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:19.776151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:19.777842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 57/113] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:19.778833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.819899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.820778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.821283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.821747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.832974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0db4a283-4a73-4c38-a8ec-6d144711cb90 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:19.838718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:19.839558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:19.840335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:19.842806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 57/114] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:19.844481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963583e3-6b8d-4e43-ac58-afb34764de5e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.844965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963583e3-6b8d-4e43-ac58-afb34764de5e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.845480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963583e3-6b8d-4e43-ac58-afb34764de5e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.846992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 58/115] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:19.848135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963583e3-6b8d-4e43-ac58-afb34764de5e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.852913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.903878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d9dac4-c769-48a1-9b8d-848f4310c2d6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:19.914969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:19.942036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:19.942683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:19.942973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:19.943251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:19.943929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:19.944236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:19.945645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:19.954558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 58/116] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:19.955391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:19.973574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963583e3-6b8d-4e43-ac58-afb34764de5e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.013921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.015008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.015379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.015883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.017094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.018575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af6f7d-5e64-4d97-ac6d-22b9d0ac8a95 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.020216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 59/117] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:20.021328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.021613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.021892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.044089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:20.045392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:20.099936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b340dd6-704c-4f3d-a86a-3f66c76b905c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.122148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.138275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 60/118] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:20.138275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.151487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.152099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.152383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.152655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.153876 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:20.154180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.156499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.169841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 59/119] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:19 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:20.171047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.218224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.256003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62e44ae3-0215-4a1e-8f4d-c3b149910638 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.299277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b3da0d-3e04-41ca-8d59-339911f37f01 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.299778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.300131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.300410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.300677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.315362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:20.321118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:20.321959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 61/120] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/roles/member => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:20.323151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.340965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.341605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.342138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.342451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] PATH_INFO: `/v3/users/f4ca822b206749fd8ea34f9bf8a3e277` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.362194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] RBAC: Authorizing `identity:get_user(user_id=f4ca822b206749fd8ea34f9bf8a3e277)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.364471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.366115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 60/121] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/users/f4ca822b206749fd8ea34f9bf8a3e277 => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:20.366908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.392513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.444541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adffddcd-8bf8-401b-be39-898ad30c1345 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.469746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.480423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd59b14-903d-4d86-af6d-2f263b3659bf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.481502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.482165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.482468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.482742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/01e05990467f46b9b8ba90e062973118/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.486423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, user_id=01e05990467f46b9b8ba90e062973118, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.487941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.501490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 62/122] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:20 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/01e05990467f46b9b8ba90e062973118/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:20.507498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.529539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.531641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.532213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.532685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.534986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.536735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.539740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 61/123] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:20.540950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.659278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71bc090c-63ff-4320-8498-512a8b1f4579 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.665868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366db352-6dee-4142-95fe-0a8e43340b2e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.684201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.704266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.704895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.705182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.705458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.706137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:20.706502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.707893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.713698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.714437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.714722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.714996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.719107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 63/124] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:20.720314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.750932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:20.798685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:20.800785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:20.860562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efab065-1636-4d7c-9a9c-9b156a14c299 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:20.897567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:20.898230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:20.898802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:20.899074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:20.911522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:20.917096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:20.917958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:20.919180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:20.924139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 64/125] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:20.925104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.941311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 62/126] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 401 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:20.946967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:20.970467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.017202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.032121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63e2f35-de3f-489f-a594-33a1ec068d0f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.063107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1189ea3e-94b7-4d55-acc4-9f899ff10c9c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.067194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.067841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.068124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.068403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.069600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.071101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.072938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 65/127] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.073485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.105194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.105835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.106360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.106841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.108116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.110438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.112704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 63/128] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:20 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.113492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.185916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17174d3-a41e-4e4b-9da7-c5105069e598 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.203425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.231267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.231602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a275c-dd99-4560-9103-7c44aca31af9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.231895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.232186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.232467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.233374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:21.233735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.236443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.250708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 66/129] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.251605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.267244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.277969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.278647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.278930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.279206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.281257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.282803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.285090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 64/130] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.285707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.393013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3baf1e6-14be-4b5f-b0dc-6eac8543992a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.409509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cab754-1ee5-4d64-8f3c-83fe28300a29 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.441885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.443589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.444372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.444857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/0674c5f80c3443de9ee43f655c58e809/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.447162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, user_id=0674c5f80c3443de9ee43f655c58e809, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.447973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.448630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.448915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.449194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] PATH_INFO: `/v3/projects/55852f8e0358419196fa1f5337bdc887` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.449799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.450420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] RBAC: Authorizing `identity:get_project(project_id=55852f8e0358419196fa1f5337bdc887)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.453286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.458821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 65/131] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/projects/55852f8e0358419196fa1f5337bdc887 => generated 312 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.458821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.472758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 67/132] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:21 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/0674c5f80c3443de9ee43f655c58e809/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 221 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:21.474380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.516819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.573382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.610033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ae6845-ba3c-4307-a18f-de36ccd2d4fd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.617712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e84484d-4a0c-4bb7-b989-da536c152a04 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.652184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.653171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.653675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.654224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.655581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.656098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=f4ca822b206749fd8ea34f9bf8a3e277) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:21.656309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.656577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.656849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.657212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.658045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.659134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.659531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.661292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 66/133] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.661870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.671617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 68/134] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/role_assignments?user.id=f4ca822b206749fd8ea34f9bf8a3e277&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=248695be26f54d9d9e20d976c4890f7d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:21.673525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.790056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaebd73f-aa1c-4619-9d88-2bc53ab53d1b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.794612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.802193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6712b22b-d76a-4b1d-b497-772b47a21129 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:21.824341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:21.831927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.832616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.832903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.833189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] PATH_INFO: `/v3/projects/55852f8e0358419196fa1f5337bdc887/users/0674c5f80c3443de9ee43f655c58e809/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.835291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=55852f8e0358419196fa1f5337bdc887, user_id=0674c5f80c3443de9ee43f655c58e809, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.836771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e77d5067-cced-4ad4-8eeb-3210a3ed56e3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.843378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:21.844041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.844321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.844592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.845813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:21.847355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049a46ef-69be-48c0-a9d5-bd485b22d128 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:21.849043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 69/135] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:21 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:21.850430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e72d-3825-4486-bb40-de9d9a13eeba None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.850719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e72d-3825-4486-bb40-de9d9a13eeba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.850994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1744e72d-3825-4486-bb40-de9d9a13eeba None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.852225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 70/136] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:21.856700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 67/137] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:21 2022] PUT /identity/v3/projects/55852f8e0358419196fa1f5337bdc887/users/0674c5f80c3443de9ee43f655c58e809/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:21.861060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e72d-3825-4486-bb40-de9d9a13eeba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:21.863465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:21.863750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:21.864025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:21.907938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:21.911101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:22.053652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 68/138] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:22.083946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1744e72d-3825-4486-bb40-de9d9a13eeba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:22.130218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:22.131305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:22.131785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:22.132252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/0674c5f80c3443de9ee43f655c58e809/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:22.141100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, user_id=0674c5f80c3443de9ee43f655c58e809, role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:22.142897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:22.158972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 71/139] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:21 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/0674c5f80c3443de9ee43f655c58e809/roles/793acbb060a2436192eb2c7413739e83 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:22.507453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:22.620059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec69d413-545e-40d7-8633-7aad15b214d4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:22.644224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:22.661319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:22.661971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:22.662522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:22.663200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:22.664333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:22.665792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3ca6ac-726f-4f5b-abd8-c833b55d9da5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:22.667419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 69/140] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:22.673411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:22.719620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:22.819014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e2b4c5d-e8fd-4ef5-84f9-846e8c7be413 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:22.857254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:22.858823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:22.859313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:22.859774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:22.860645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:22.862305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0736dcf6-f330-4116-bfc1-6402e6599c24 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:22.885804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 72/141] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:22 2022] POST /identity/v3/users => generated 283 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:23.127931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481511b-938a-4d91-a961-256614d890b8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:23.128360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481511b-938a-4d91-a961-256614d890b8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:23.128562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1481511b-938a-4d91-a961-256614d890b8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:23.130234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 70/142] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:23.135538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:23.136093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:23.136561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:23.160473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:23.162082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:23.257983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 73/143] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:23.571367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481511b-938a-4d91-a961-256614d890b8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:23.701404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1481511b-938a-4d91-a961-256614d890b8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:23.745341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:23.746385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:23.746863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:23.747329 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:23.761529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:23.766831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:23.767691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:23.768454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:23.770888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 71/144] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:23.777182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:23.789997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:23.943811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f7ed11-0709-4f7e-96e3-17929b5e1a7e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:23.990630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:23.991257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:23.991538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:23.991806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:23.992484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:23.992787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:23.994166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:24.006946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 74/145] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:24.013386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:24.104012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:24.173857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4310321-4dc7-414a-a5f0-b1105bfb224d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:24.219987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:24.220558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.220844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.221122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] PATH_INFO: `/v3/users/01e05990467f46b9b8ba90e062973118` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.226974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] RBAC: Authorizing `identity:get_user(user_id=01e05990467f46b9b8ba90e062973118)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:24.228479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673dc393-dfb2-429f-8695-c4c60640101f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:24.230126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 72/146] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:24 2022] GET /identity/v3/users/01e05990467f46b9b8ba90e062973118 => generated 247 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:24.236215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:24.355422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:24.394970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f835b3-4830-47ea-9196-f14e5f3e5a2a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:24.446368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:24.446994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.447277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.447553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.503525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01a57a-5b42-4956-a4b4-6482adf173c2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.503708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01a57a-5b42-4956-a4b4-6482adf173c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.504041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf01a57a-5b42-4956-a4b4-6482adf173c2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.505350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 73/147] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:24.510649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a3bd-cf87-4ca8-9216-19c3ee3389cc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.510937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a3bd-cf87-4ca8-9216-19c3ee3389cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.511217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba52a3bd-cf87-4ca8-9216-19c3ee3389cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.546468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:24.547733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ba52a3bd-cf87-4ca8-9216-19c3ee3389cc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:24.548017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:24.548978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba52a3bd-cf87-4ca8-9216-19c3ee3389cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:24.660428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 74/148] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:24.661530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52914d5c-4a5b-4dcd-8e1b-61bf640298b5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.661812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52914d5c-4a5b-4dcd-8e1b-61bf640298b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.662085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52914d5c-4a5b-4dcd-8e1b-61bf640298b5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.663483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 75/149] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:24.670219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.670706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.670982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.680706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:24.709283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:24.710507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:24.746442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 75/150] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 511 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:24.753912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:24.794266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:24.862631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 76/151] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:24.902856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39af8e34-e0a2-40c8-8e74-c275e6600299 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:24.955972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:24.956630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.956913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.957187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.958391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:24.959845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e603c024-e2ec-4a86-bce7-f9095d736337 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:24.961798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 76/152] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:24 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:24.968266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:24.985430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91806bf-cf42-43d3-a258-c785b550d39e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.985752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91806bf-cf42-43d3-a258-c785b550d39e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.985988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91806bf-cf42-43d3-a258-c785b550d39e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:24.987534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 77/153] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:24.993508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:24.993788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:24.994059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.038984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:25.040133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:25.130671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:25.188884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9925f9a-11b8-4dab-988a-1c902bbcfcd1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:25.211960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 78/154] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:25.212819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:25.255552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:25.256513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.257080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.257633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.258691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=01e05990467f46b9b8ba90e062973118) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:25.259189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:25.260897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:25.271565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 77/155] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:24 2022] GET /identity/v3/role_assignments?user.id=01e05990467f46b9b8ba90e062973118&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:25.329389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:25.388321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0255e2f5-0164-48b3-9963-231cddcde19a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:25.412554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:25.426485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:25.427166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.427425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.427694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.439949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dfe62f64-e476-4404-9aba-a9511ad50707 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:25.445201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:25.446100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:25.446907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 79/156] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:25.448326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f214-5400-438e-924b-86888fcb3705 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.448326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f214-5400-438e-924b-86888fcb3705 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.448326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc5f214-5400-438e-924b-86888fcb3705 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.449579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 80/157] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:25.456018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.456367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.456703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.476463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:25.479979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:25.511470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:25.531522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4bc76fd-aa99-4de5-aa81-ed6563a7b775 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:25.554224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 81/158] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:25.555298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:25.650987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:25.651681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.651962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.652236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.672130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:25.678182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:25.679164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:25.679932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:25.681704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecbfdc46-58e0-493f-947f-f49118d3404d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:25.687820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 78/159] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 359 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:25.693927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:25.719240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:25.719998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.720308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.720719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.721402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:25.721751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:25.723314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:25.733895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 82/160] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:25.744035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:25.778609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:25.862654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc2cf7-ea52-400a-9794-6a8dd7a0ac99 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:25.907237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:25.907825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:25.908065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:25.908357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:25.909548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:25.911038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:25.913027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 83/161] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:25.913803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.006733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.012228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3790e6-ae1c-4f47-b9ad-ae52ef5fca43 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.046066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.080067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.080711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.080870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ff1a47-41b1-4dcf-9c4d-12aa746cefee None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.081004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.081273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.081925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:26.082223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:26.083892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:26.099707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 79/162] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:26.102365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.161437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.161995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.162297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.162761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.174653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:26.180602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:26.181493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:26.182289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 84/163] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:26.182964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.305843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13504bc1-a667-4a42-9ba2-5168309c6e44 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.346521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.347159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.347436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.347705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.348855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:26.351012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:26.352829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 85/164] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:26.353742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.355137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-443cc270-69a5-4c3a-b883-4adc672beb2f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.389181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.461368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.461918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.462199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.462500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.481542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dbc976-bac9-4469-9f01-f6e6d788f35c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.520658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.521279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.521561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.521831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.533249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:26.538685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:26.539565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:26.540352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 86/165] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:26.540991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.546187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.650723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.662077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d75764-1df0-47c0-8b42-e412f7feb4b7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.673320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:26.674833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:26.727415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.728729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.729013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.729292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.729961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:26.730278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:26.732313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:26.752012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 87/166] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:26.752856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.871318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de65bb24-b5e6-4ba3-9fbb-967ce3a13b77 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:26.882339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 80/167] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 782 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:26.883051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:26.908803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:26.909526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:26.909747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:26.910070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:26.921553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:26.983851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:26.985528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.019109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.074119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4867166-e1b3-495b-9237-dd917590cd58 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.113190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.113817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.114099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.114291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 88/168] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:27.114735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.115158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.115412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:27.115718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.117380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.128203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 81/169] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.129056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.175173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.245932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9f5407-56a5-4266-83c5-1c86077f1b8a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.255110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc31a68-be08-4692-bb9d-d3d329962b04 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.273008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.285518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.286162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.286486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.286830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] PATH_INFO: `/v3/projects/55852f8e0358419196fa1f5337bdc887` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.287974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] RBAC: Authorizing `identity:get_project(project_id=55852f8e0358419196fa1f5337bdc887)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.289449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27143e7f-13f8-47bd-b3e9-81e36ee4a677 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.296733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 82/170] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/projects/55852f8e0358419196fa1f5337bdc887 => generated 312 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.297981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed723-4c12-425c-8c3b-8010d2cfb36b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.298281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed723-4c12-425c-8c3b-8010d2cfb36b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.298582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01aed723-4c12-425c-8c3b-8010d2cfb36b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.300133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 83/171] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:27.301302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aed723-4c12-425c-8c3b-8010d2cfb36b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.303929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.305072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.306756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.307238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] PATH_INFO: `/v3/users/f4ca822b206749fd8ea34f9bf8a3e277` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.308719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] RBAC: Authorizing `identity:get_user(user_id=f4ca822b206749fd8ea34f9bf8a3e277)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.310412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.313150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 89/172] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/users/f4ca822b206749fd8ea34f9bf8a3e277 => generated 289 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.314780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.417521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01aed723-4c12-425c-8c3b-8010d2cfb36b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.455912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.456517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.456798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.457072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.458213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.459297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52488e-c70e-4b7e-ad95-7bc68e92b93d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.459708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.461692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 84/173] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.462584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.472584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.515598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.517232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.518122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.518933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] PATH_INFO: `/v3/users/0674c5f80c3443de9ee43f655c58e809` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.520577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] RBAC: Authorizing `identity:get_user(user_id=0674c5f80c3443de9ee43f655c58e809)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.523051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae6ab9c3-f05c-4f7f-88a4-bbc05acef0a4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.525869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 90/174] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/users/0674c5f80c3443de9ee43f655c58e809 => generated 275 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.527334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.527834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.528313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.540887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.555371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:27.557883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:27.594272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24aeb65c-809e-4a11-bc7f-db6f93857a59 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.633417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.634042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.634347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.634663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.635597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=55852f8e0358419196fa1f5337bdc887, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:27.635711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.637626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.643782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 91/175] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:27.644632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 85/176] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=55852f8e0358419196fa1f5337bdc887&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 608 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.644744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.645575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.761592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b0b320-abb0-4a28-b23d-8e34f6ebca70 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.770726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.776848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8080347-07a0-4781-97f5-54dd483144a2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:27.805852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.806511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:27.807229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.807550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.807856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.809152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.812072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.815599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 92/177] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.816096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.835714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:27.836328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.836603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.836869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.837596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:27.837901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:27.839375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681fb370-40a9-4b69-8108-0d84447a9e85 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:27.847820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 86/178] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:27.849027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b79a7-a137-470b-8e69-f14a3ccacfd4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.849304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b79a7-a137-470b-8e69-f14a3ccacfd4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.849572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395b79a7-a137-470b-8e69-f14a3ccacfd4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.851656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 87/179] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:27.852725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25914c7-e1c0-447b-af28-8e3cf607579c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:27.853052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25914c7-e1c0-447b-af28-8e3cf607579c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:27.853384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25914c7-e1c0-447b-af28-8e3cf607579c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:27.854722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 88/180] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:27.855259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25914c7-e1c0-447b-af28-8e3cf607579c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:27.965338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b7cc62-5792-4ff0-85fa-290fbf0649c0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.003000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25914c7-e1c0-447b-af28-8e3cf607579c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.008757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.009657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.010134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.010609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.049284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.049880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.050156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.050461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/f4ca822b206749fd8ea34f9bf8a3e277/roles/248695be26f54d9d9e20d976c4890f7d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.052468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, user_id=f4ca822b206749fd8ea34f9bf8a3e277, role_id=248695be26f54d9d9e20d976c4890f7d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.053921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80551797-c096-4ca4-a947-235d4d97eb90 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.064312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.069916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 89/181] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:27 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/users/f4ca822b206749fd8ea34f9bf8a3e277/roles/248695be26f54d9d9e20d976c4890f7d => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:28.070998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb08fd-80fe-4705-8a03-b2bc6e4593eb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.071354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb08fd-80fe-4705-8a03-b2bc6e4593eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.071627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdb08fd-80fe-4705-8a03-b2bc6e4593eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.091985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ccdb08fd-80fe-4705-8a03-b2bc6e4593eb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:28.095494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ccdb08fd-80fe-4705-8a03-b2bc6e4593eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:28.099407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.100485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.159985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.187865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 90/182] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:28.189315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.189658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.189934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.209831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:28.211074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:28.248213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 93/183] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:28.249743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.310125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 91/184] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:28.311974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.344365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.378347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f81c5d3-591f-4d32-8e67-4d8e49b0418d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.437373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6857df3-189c-4e6d-8361-cae568ab8383 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.447983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.477240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.477874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.478151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.478508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.479901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.481412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.484185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 92/185] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:28.487510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.488160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.488442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.488710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.490554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.516091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:28.521289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:28.522274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:28.523409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 94/186] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:28.524133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.631603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599ff609-fdd9-4481-836d-a4cdfc3c8604 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.661435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f482b830-211b-4752-a5cc-0dca7dd06042 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.670857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.671469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.671750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.672017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.672755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=0674c5f80c3443de9ee43f655c58e809) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:28.673061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.674544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.681031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 93/187] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/role_assignments?user.id=0674c5f80c3443de9ee43f655c58e809&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=793acbb060a2436192eb2c7413739e83 => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:28.681903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.704976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.705689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.706108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.706423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.708874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.722425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:28.727645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:28.728624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:28.729444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:28.734843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 95/188] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:28.735916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.753856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:28.846663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e723b497-fea1-4c49-a2de-c093efff7075 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.864193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661d7dad-a149-4856-a388-c8d35dd136ce None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:28.892583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.893418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.893906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.894428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.895321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:28.895834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.897307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.905273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:28.906119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:28.906462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:28.906735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:28.908721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:28.909540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 94/189] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:28.910626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:28.910753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.911933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 96/190] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:28.916869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:28.984292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.027508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71895a38-f37b-4cb0-987c-765da3bfcd04 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.029138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.032573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d205a298-9485-49f3-a35d-a48b88d1106e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.065701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.066395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.066609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.066884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.067543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:29.067845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.069216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.069926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.070936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.071703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.072125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.082996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 95/191] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:29.082996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.087139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.088035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.089812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 97/192] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:28 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:29.090475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.206366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafce305-ff0e-426a-8cf0-1621b74ac6e8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.206547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ac1840-439e-43d9-8ccd-1a2a8bb20309 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.234575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.244779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.245400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.245679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.246048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.246835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.248328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.256757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.256757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.256757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.256757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.257987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.259711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.267924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 98/193] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:29.269276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.282654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 96/194] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:29 2022] POST /identity/v3/users => generated 297 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:29.283529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.310414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.456057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ec6f42-4825-4577-93f8-c0677f0883de None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.494574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620b649-deaa-4c22-8d6e-a93de21ba30a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.505287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.505916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.506196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.506495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.559905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.560995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.562673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.562960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.618151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.636485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.638416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.661130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.663034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.695561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.786310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 99/195] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 517 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:29.791730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.804475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 97/196] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:29.810146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.892428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.906241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f5f95a-c77e-4b6b-b598-1e233c01e088 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.924949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503937ce-cee6-4af3-af95-4d95ae5efa73 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:29.942463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.943102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.943386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.943657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.944802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.946283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.947961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:29.948296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 100/197] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:29.962293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:29.982052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:29.983066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:29.983504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:29.983930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:29.987458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:29.989187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:29.994644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 98/198] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:30.001367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:30.060565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95203a-6984-4343-8402-d7a91d17a9fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:30.110431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:30.111046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:30.111333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:30.111606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:30.112351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=1670e97ee2ff4de1a8a1a7af56f92cb2) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:30.112659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:30.114108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3229ed40-782b-4ac1-a792-2feab5d4e287 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:30.120511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 101/199] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:29 2022] GET /identity/v3/role_assignments?user.id=1670e97ee2ff4de1a8a1a7af56f92cb2&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:30.199687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e80fdc6-3726-4ed4-8ef4-636db2b32626 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:30.253446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:30.254457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:30.254959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:30.255432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:30.256379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=1670e97ee2ff4de1a8a1a7af56f92cb2) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:30.256892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:30.258583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26610f15-02d8-4ec3-acc1-285fc8c5c530 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:30.267535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 99/200] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:30 2022] GET /identity/v3/role_assignments?user.id=1670e97ee2ff4de1a8a1a7af56f92cb2&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=793acbb060a2436192eb2c7413739e83 => generated 259 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:30.360258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b052-a5f5-450c-a7fc-438be49c6c82 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:30.360591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b052-a5f5-450c-a7fc-438be49c6c82 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:30.360811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb4b052-a5f5-450c-a7fc-438be49c6c82 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:30.362509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 102/201] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:30.368057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:30.368342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:30.368617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:30.389939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:30.391242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:30.445468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:30.468300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 100/202] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:30.785910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4b052-a5f5-450c-a7fc-438be49c6c82 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:30.814736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:30.935503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb4b052-a5f5-450c-a7fc-438be49c6c82 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:30.980969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:30.981604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:30.981883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:30.982151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:30.992774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:30.998042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:30.999676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:31.000504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 103/203] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:31.011345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:31.170916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8bdd408-e30f-489f-b76a-1a417254f80e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:31.215044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:31.215678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:31.215953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:31.216222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:31.216996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:31.217300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:31.218844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:31.235288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 101/204] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:31.240561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:31.360860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:31.385645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc55ddc8-e64a-42f8-b7e6-583ccb14ee94 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:31.444505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:31.445074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:31.445351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:31.445626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] PATH_INFO: `/v3/users/f4ca822b206749fd8ea34f9bf8a3e277` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:31.446804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] RBAC: Authorizing `identity:get_user(user_id=f4ca822b206749fd8ea34f9bf8a3e277)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:31.448234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:31.449991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 104/205] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:31 2022] GET /identity/v3/users/f4ca822b206749fd8ea34f9bf8a3e277 => generated 289 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:31.456190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:31.514665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:31.586932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857ef4d8-df6f-48dc-94bf-f98d4cd7afa4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:31.636801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:31.637417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:31.637690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:31.637959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:31.737602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:31.739056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a1424-a82c-4a78-bd53-6e0ef6bf8d16 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:31.793463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:31.853544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 102/206] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:31.863375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:31.983715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3748f689-6163-42ad-89d9-e3dbe8e31bc6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:31.993941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e298f79-1559-4541-9ecd-d79d3618a5c5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:31.994219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e298f79-1559-4541-9ecd-d79d3618a5c5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:31.994433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e298f79-1559-4541-9ecd-d79d3618a5c5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:31.996049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 103/207] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:32.001651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ce8-0156-432d-8e23-31caaa735829 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.001938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ce8-0156-432d-8e23-31caaa735829 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.002219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-311c1ce8-0156-432d-8e23-31caaa735829 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.026994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:32.027932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.028382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.028852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.030928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:32.032561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:32.034903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 105/208] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:31 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:32.041065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:32.044107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-311c1ce8-0156-432d-8e23-31caaa735829 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:32.046622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-311c1ce8-0156-432d-8e23-31caaa735829 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:32.075013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:32.167317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 104/209] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:32.221623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd078f6-e381-40b1-ab91-8541fae19ea0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:32.266570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:32.267952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.268425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.268893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.269811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237, user.id=f4ca822b206749fd8ea34f9bf8a3e277) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:32.270348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:32.271986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9ada4e-9fc6-46fe-91d0-a1dd8bda9a56 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:32.279099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 106/210] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:32 2022] GET /identity/v3/role_assignments?user.id=f4ca822b206749fd8ea34f9bf8a3e277&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=248695be26f54d9d9e20d976c4890f7d => generated 608 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:32.434411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efab4f5-5afc-4255-a214-2dac7acd59e6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.435094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efab4f5-5afc-4255-a214-2dac7acd59e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.436157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efab4f5-5afc-4255-a214-2dac7acd59e6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.437808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 105/211] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:32.444469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd4c9-edec-4c1b-b8c6-ab58ec43ac90 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.445006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd4c9-edec-4c1b-b8c6-ab58ec43ac90 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.445493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87cd4c9-edec-4c1b-b8c6-ab58ec43ac90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.469933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b87cd4c9-edec-4c1b-b8c6-ab58ec43ac90 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:32.472136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b87cd4c9-edec-4c1b-b8c6-ab58ec43ac90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:32.505734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efab4f5-5afc-4255-a214-2dac7acd59e6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:32.559933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 107/212] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:32.561693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c353a-bc76-411a-a974-21a003b2caa6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.562043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c353a-bc76-411a-a974-21a003b2caa6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.562312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900c353a-bc76-411a-a974-21a003b2caa6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.571735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 108/213] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:32 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:32.577368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365acb-b11b-4116-b551-0e82e9409057 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.577607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365acb-b11b-4116-b551-0e82e9409057 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.577836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f365acb-b11b-4116-b551-0e82e9409057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.600852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:32.606024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5f365acb-b11b-4116-b551-0e82e9409057 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:32.608095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5f365acb-b11b-4116-b551-0e82e9409057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:32.635167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efab4f5-5afc-4255-a214-2dac7acd59e6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:32.635659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:32.676620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:32.677368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.677642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.677907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.691817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:32.697237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:32.698141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:32.698994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:32.699700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 109/214] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:32.699700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16ae8-edee-4bf3-a631-5a6d939f4768 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.699700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16ae8-edee-4bf3-a631-5a6d939f4768 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.699700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba16ae8-edee-4bf3-a631-5a6d939f4768 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.700503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 110/215] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:32.705577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 106/216] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:32.706684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.706969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.707240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.716110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:32.729933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:32.731785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:32.829310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 111/217] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:32.837419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3fc7486-099d-4dc9-bd6b-377f52edde78 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:32.856385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:32.884088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:32.885159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:32.885931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:32.886877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:32.887803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:32.888453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:32.890646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:32.892369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:32.904215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 107/218] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:32.909587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:32.931646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:32.965655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ea2d8d-9171-431e-859d-ed3f2d724fc5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.001411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.002424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.002879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.003288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.013775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:33.019308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:33.020158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:33.020938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 112/219] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:33.021616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.044854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6adc21e-04e2-44dc-8f31-ac434f0458dd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.080972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.081526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.081805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.082078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] PATH_INFO: `/v3/users/947b64c3bee445379f566c2f1343c813` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.097014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] RBAC: Authorizing `identity:get_user(user_id=947b64c3bee445379f566c2f1343c813)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.098608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.100274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 108/220] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:32 2022] GET /identity/v3/users/947b64c3bee445379f566c2f1343c813 => generated 297 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:33.101028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.144673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70345c64-4611-433b-9cca-874778c5c8f7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.163035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.174333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.180504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.181176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.181456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.181720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.182850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.184285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.190390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 113/221] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.190390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.211835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792e1e41-2d6b-49cb-ab1e-484cb050acba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.246592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.247207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.247484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.247759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.248920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:33.249222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.250642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.258765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 109/222] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.259726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.288805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53859705-5837-4ece-b580-ce23809b88fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.322816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.323413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.323686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.323950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.363805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0afb1b1a-3db6-472c-b080-61bf282c3c50 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.379903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.389982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.394214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.395691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.399995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.400621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.400892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.401158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.411274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:33.417272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:33.418241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 110/223] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:33.419301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.498869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 114/224] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:33.499969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.524721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766c59b2-c986-400e-b986-4a1b7bd7424a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.559117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.559716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.559989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.560256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.561673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.563144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.584743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 111/225] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:33 2022] POST /identity/v3/users => generated 297 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:33.585554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.607333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.612482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b041fd08-00f2-46f5-b17f-fdba153fdc88 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.633233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.655208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.656840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.657378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.657811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.659354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.661277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.663667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 115/226] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.665009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.717048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac99dd6f-9c01-42f0-a56b-7c8b92d606b9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.755801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.756408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.756682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.756955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.757618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:33.757909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.759910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.770951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 112/227] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.771796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.782414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b64ee3ad-7afd-4732-ae0f-7524b466aa9d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.829631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.830682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.830682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.830873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.832061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.834116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.837988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 116/228] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.838714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.896196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.909805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:33.916819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ad4d30-a226-4de1-a927-e58432347be7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:33.958855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:33.959490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:33.959763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:33.960032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:33.961430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:33.962914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:33.964722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 113/229] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:33.965308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:33.980014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eab2a92-6bb2-4520-99a2-979218a4d8fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:34.018312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:34.019060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:34.019345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:34.019724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:34.020885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:34.022470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:34.024720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 117/230] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:34.025459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:34.088146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afaa1165-8b91-423b-82ae-72dbc3cc7845 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:34.125376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:34.126065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:34.126374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:34.126708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:34.127636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=947b64c3bee445379f566c2f1343c813) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:34.127944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:34.129767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:34.136365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 114/231] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:33 2022] GET /identity/v3/role_assignments?user.id=947b64c3bee445379f566c2f1343c813&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:34.142800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:34.156812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacf9b7e-134a-40ae-9988-e8206e76ce7f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:34.158348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:34.213422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:34.214107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:34.214416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:34.214691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/users/1670e97ee2ff4de1a8a1a7af56f92cb2/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:34.221081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, user_id=1670e97ee2ff4de1a8a1a7af56f92cb2, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:34.221531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:34.222681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:34.255156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 118/232] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:34 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/users/1670e97ee2ff4de1a8a1a7af56f92cb2/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:34.319008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc72-cfc9-4fb9-8e78-aa18d31dea28 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:34.379660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:34.380629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:34.381137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:34.381652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:34.383969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:34.385897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e29285c-6b68-4773-aa9c-18493df213cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:34.388309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 115/233] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:34 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:34.394483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:34.504658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71c7d22-e946-4755-b57d-f1a92f8fd392 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:34.540771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:34.541833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:34.542308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:34.542750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/users/1670e97ee2ff4de1a8a1a7af56f92cb2/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:34.545729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, user_id=1670e97ee2ff4de1a8a1a7af56f92cb2, role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:34.547410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363ce40e-a441-49e6-9a81-d6dcce9e3a5e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:34.564105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 119/234] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:34 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/users/1670e97ee2ff4de1a8a1a7af56f92cb2/roles/793acbb060a2436192eb2c7413739e83 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:36.038014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ed5e1-749c-4084-8468-cea35a56baf6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.039053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ed5e1-749c-4084-8468-cea35a56baf6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.039584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447ed5e1-749c-4084-8468-cea35a56baf6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.041368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 116/235] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:36.046629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a0b6-2373-4ccd-b200-d6a319f824cf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.047451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a0b6-2373-4ccd-b200-d6a319f824cf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.047952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb18a0b6-2373-4ccd-b200-d6a319f824cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.071356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abd40b-998b-4443-b8eb-7cca5a6c106d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.071954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abd40b-998b-4443-b8eb-7cca5a6c106d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.072416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55abd40b-998b-4443-b8eb-7cca5a6c106d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.073938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 117/236] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:36.076153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eb18a0b6-2373-4ccd-b200-d6a319f824cf None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:36.077660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eb18a0b6-2373-4ccd-b200-d6a319f824cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:36.079267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f003-596a-4b0b-88ed-459f529a61d4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.079803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f003-596a-4b0b-88ed-459f529a61d4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.080277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4640f003-596a-4b0b-88ed-459f529a61d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.110849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4640f003-596a-4b0b-88ed-459f529a61d4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:36.112888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4640f003-596a-4b0b-88ed-459f529a61d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:36.122625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:36.187345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 120/237] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:36.256664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 118/238] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:36.539818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72034f2-9d0f-4636-9b05-bd8c9c172b24 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.540531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72034f2-9d0f-4636-9b05-bd8c9c172b24 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.541003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72034f2-9d0f-4636-9b05-bd8c9c172b24 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.542832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 121/239] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:36.548543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.549047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.549523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.559420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72034f2-9d0f-4636-9b05-bd8c9c172b24 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:36.583361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:36.584628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:36.604314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:36.669377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 119/240] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:36.671181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:36.680744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f72034f2-9d0f-4636-9b05-bd8c9c172b24 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:36.730314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:36.731323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.731791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.732358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.739934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:36.739934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df14e8d-1ab1-4575-8f69-6e948b89dcfb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:36.741325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 122/241] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:36.742504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2f75-d556-413c-a6ac-4c47ab0433d0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.742785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2f75-d556-413c-a6ac-4c47ab0433d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.743051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ac2f75-d556-413c-a6ac-4c47ab0433d0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.744356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 123/242] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:36.753512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac2f75-d556-413c-a6ac-4c47ab0433d0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:36.782751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:36.795374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae1ce2a-6bb6-44ac-ab05-bc8210668b8f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:36.837106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:36.837883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.838289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.838651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.850367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3b0f7408-4fb6-43e2-aeb0-8b312bb92a2c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:36.855528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:36.856616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:36.858275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:36.860189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 120/243] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:36 2022] GET /identity/v3/roles/reader => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:36.862182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.862588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.862952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.884023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:36.886183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:36.901665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:36.910291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ac2f75-d556-413c-a6ac-4c47ab0433d0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:36.952263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:36.952710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:36.953587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:36.953949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:36.954736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:36.956266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:36.981403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 124/244] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:36 2022] POST /identity/v3/users => generated 293 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:36.990449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 121/245] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:36.991522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:36.993824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.108645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0210b95d-eee4-42e4-bd4f-1b74f851ee10 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.146078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.153349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.154726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.156169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.156674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.168964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:37.171777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48e8adc0-8655-48d3-949a-c58fbacb0fd2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:37.175480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:37.176367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:37.177160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:37.179590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 122/246] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:37.187881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.194864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.213557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.214175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.214490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.214762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.215439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:37.215740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:37.217358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:37.227372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 125/247] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:36 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:37.236060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.326642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45ef27d-14de-4d84-8f46-e5aea3c9a16c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.361504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bde11c2-4ebc-48ff-a44f-2f5a4ba9d9fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.366167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.366811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.367085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.367355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.368030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:37.368714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:37.370130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c1dcf-e22f-46c5-b967-9806b25cacf6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:37.383144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 123/248] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:37.384416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c10ee-1e57-416f-96f9-0269d79573d9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.384698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c10ee-1e57-416f-96f9-0269d79573d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.385030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510c10ee-1e57-416f-96f9-0269d79573d9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.386885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 124/249] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:37.391153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c10ee-1e57-416f-96f9-0269d79573d9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.403927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.405137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.405380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.405642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] PATH_INFO: `/v3/users/6b0354746a8447a698a3203af8eb0526` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.414533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.423682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] RBAC: Authorizing `identity:get_user(user_id=6b0354746a8447a698a3203af8eb0526)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:37.425910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:37.428205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 126/250] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/users/6b0354746a8447a698a3203af8eb0526 => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:37.429390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.479075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.531197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510c10ee-1e57-416f-96f9-0269d79573d9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.557079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62bbaf4-3e93-4329-b2cc-55d2736700b6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.572466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.573241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.573589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.573929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] PATH_INFO: `/v3/users/947b64c3bee445379f566c2f1343c813` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.575175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] RBAC: Authorizing `identity:get_user(user_id=947b64c3bee445379f566c2f1343c813)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:37.576731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057e25af-0300-4536-8564-84e01b7461d0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:37.578808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 125/251] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/users/947b64c3bee445379f566c2f1343c813 => generated 297 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:37.580185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.580553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.580894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.599762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.600409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.600690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.601058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.601980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:37.603325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:37.615550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:37.622234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:37.623718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:37.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:37.626017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 127/252] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:37.626017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.687063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 126/253] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:37.687927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.715701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.766809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:37.811131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a81c6e6-7a17-4078-b676-6bb917ec9819 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.822961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6004d2c6-83d5-43bd-ac49-0c4848a6880b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:37.858827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.859793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.860241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.860681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.861972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:37.863620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:37.865412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 127/254] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:37.866538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:37.886238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:37.887075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:37.887362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:37.887631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:37.975067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f00012-faf4-482b-80da-3512626b5898 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.006643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.015678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.017260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.017930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.019092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.020118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:38.021592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.024324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.041424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 128/255] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:38.043734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.045680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.047606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.116637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.158300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27fe2b1c-6f18-437c-b08a-eb2868f41704 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.184789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 128/256] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 559 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:38.186921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.196186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.196850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.197128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.197401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/users/947b64c3bee445379f566c2f1343c813/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.199507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, user_id=947b64c3bee445379f566c2f1343c813, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.201115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.215533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 129/257] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:38 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/users/947b64c3bee445379f566c2f1343c813/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:38.217165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.289576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.300873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a44e5-909f-44c7-ad6a-f9f003a5e638 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.352529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.353190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.353469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.353737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.355523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.356990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.365051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 129/258] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:38.365950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.416960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.424438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97999994-bfbe-4f29-8043-9f522eea7bc0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.467844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.468892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.469442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.469927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.483777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:38.489815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:38.490754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:38.491553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:38.493960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 130/259] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:38.495003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.508296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f1e942-ff3e-4684-8d52-723ffbd34f7f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.546480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.547640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.547640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.547640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.549073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.550620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.552855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 130/260] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:38.553787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.618220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1638a142-6728-4aca-8403-969e93b6c6d1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.633321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.653261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.656435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.657288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.657804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.658296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.663819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2101e6a-2355-4ecf-a4a0-a740ddb0e5a9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.699765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.700295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.700624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.701436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.702360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:38.702664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.704382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.712469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 131/261] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:38.713326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.736344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.738231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.834216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedbab26-0b9e-42bc-8819-83c5058292da None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:38.846603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 131/262] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:38.847768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.870437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30901a-b001-4303-894e-383316445e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:38.871050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30901a-b001-4303-894e-383316445e1d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:38.871329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30901a-b001-4303-894e-383316445e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:38.871598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b30901a-b001-4303-894e-383316445e1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:38.872325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30901a-b001-4303-894e-383316445e1d None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=6b0354746a8447a698a3203af8eb0526) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:38.872735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30901a-b001-4303-894e-383316445e1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:38.874449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b30901a-b001-4303-894e-383316445e1d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:38.880428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 132/263] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/role_assignments?user.id=6b0354746a8447a698a3203af8eb0526&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=248695be26f54d9d9e20d976c4890f7d => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:38.881331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30901a-b001-4303-894e-383316445e1d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:38.903579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.906692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:38.968184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f83f34a-8d40-427f-a906-18cb30a8eb24 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.007308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.008390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.008869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.009337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] PATH_INFO: `/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.010925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] RBAC: Authorizing `identity:get_user(user_id=1670e97ee2ff4de1a8a1a7af56f92cb2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.012839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.014910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 132/264] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/users/1670e97ee2ff4de1a8a1a7af56f92cb2 => generated 283 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:39.020668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:39.052160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b30901a-b001-4303-894e-383316445e1d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.093502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.095182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.095668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.096131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.097735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.099506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.102345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 133/265] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:38 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:39.108776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:39.185412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:39.206146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f6ac9d-185d-4012-9ef5-1209395f59cb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.242536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.243472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.243964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.244425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.270607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:39.299748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcad6e9-75fb-405f-bf9d-2daf6adbf30a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.317558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.319419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-031bf195-229a-49db-90f2-8a6fc6a6785d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.362975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.363594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.363871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.364138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.364866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=1670e97ee2ff4de1a8a1a7af56f92cb2) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:39.365173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.366629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.382928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 134/266] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:39 2022] GET /identity/v3/role_assignments?user.id=1670e97ee2ff4de1a8a1a7af56f92cb2&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:39.425980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:39.430033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 133/267] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:39.439403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:39.566453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7852a-349b-4216-a3ae-214554fab79d None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.567116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7852a-349b-4216-a3ae-214554fab79d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.567596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba7852a-349b-4216-a3ae-214554fab79d None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.569229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 134/268] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:39.578842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd2a68-f4f5-4507-ade7-914bda8e1de0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.579453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd2a68-f4f5-4507-ade7-914bda8e1de0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.579921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fd2a68-f4f5-4507-ade7-914bda8e1de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.598227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d944b80-b751-468e-a401-27306bf8fb34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.617619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-61fd2a68-f4f5-4507-ade7-914bda8e1de0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:39.619212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61fd2a68-f4f5-4507-ade7-914bda8e1de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:39.672368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.673379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.674734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.675242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.679606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.681413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.686912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 135/269] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:39 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:39.693284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:39.726067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:39.785665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 135/270] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 04:05:39.828209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c9ca577-f2f2-41ab-bdac-1f5b81ef676b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:39.869963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:39.871046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.871544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.872007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.873470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=1670e97ee2ff4de1a8a1a7af56f92cb2) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:39.874040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:39.875704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a40a9a8-fe2f-47b4-bbc8-7c778669d181 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:39.885165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 136/271] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:39 2022] GET /identity/v3/role_assignments?user.id=1670e97ee2ff4de1a8a1a7af56f92cb2&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=793acbb060a2436192eb2c7413739e83 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:39.997229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb4cd-373d-4bda-9300-61e62e571f2a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:39.997575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb4cd-373d-4bda-9300-61e62e571f2a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:39.997818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195cb4cd-373d-4bda-9300-61e62e571f2a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:39.999466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 136/272] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:40.007950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.008252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.008494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.032495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:40.033722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:40.130452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 137/273] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:40.244394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cb4cd-373d-4bda-9300-61e62e571f2a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:40.373829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195cb4cd-373d-4bda-9300-61e62e571f2a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:40.392190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:40.420765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:40.421729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.422182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.422664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.423995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:40.425663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:40.427661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 137/274] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:40.433408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:40.449348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:40.464916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:40.538772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a8791a-f0ce-49b3-9606-4c6e76d51dd7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:40.574848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:40.575560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.576122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.576230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.576958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:40.577677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7735e-3f37-4719-a8a4-87619d4a3be2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:40.578577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:40.611456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 138/275] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:05:40 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:40.632308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:40.633409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.633935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.634436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.648751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:40.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:40.657821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:40.658678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:40.661334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 138/276] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:40.667863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:40.750059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:40.788643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be7438c2-2d3e-44ff-8eaa-e385e0661f12 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:40.829545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:40.830165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.830477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.830746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.831415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:40.831713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:40.833083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ad9fd1a-47b2-4ddc-8c77-f2692afaf4fd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:40.842713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 139/277] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:40.848409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:40.898098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:40.961629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a919965b-cf64-4c72-9772-feea18183edc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:40.985001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d499e-49b4-4651-8c3b-e9e9dd03bcfb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.985281 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d499e-49b4-4651-8c3b-e9e9dd03bcfb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.985553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252d499e-49b4-4651-8c3b-e9e9dd03bcfb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:40.987118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 140/278] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:40.993902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371501-ad64-455d-9245-49bb5c63491f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:40.994189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371501-ad64-455d-9245-49bb5c63491f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:40.994507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66371501-ad64-455d-9245-49bb5c63491f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.000340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:41.000976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.001251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.001522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] PATH_INFO: `/v3/users/fe3320dfbcde4f178c454191a9b4c94f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.016450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] RBAC: Authorizing `identity:get_user(user_id=fe3320dfbcde4f178c454191a9b4c94f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:41.017931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:41.018906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-66371501-ad64-455d-9245-49bb5c63491f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:41.019788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 139/279] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:40 2022] GET /identity/v3/users/fe3320dfbcde4f178c454191a9b4c94f => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:41.020172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66371501-ad64-455d-9245-49bb5c63491f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:41.025919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:41.110802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 141/280] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:41.141543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:41.143826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1067c271-e151-4353-bcc8-24651c897215 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:41.181146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:41.182040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.182543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.182983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.183884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, user.id=fe3320dfbcde4f178c454191a9b4c94f, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:41.184358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:41.186611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3214d92-77c7-4470-a5b4-b7d1424c03b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:41.192279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 140/281] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/role_assignments?user.id=fe3320dfbcde4f178c454191a9b4c94f&scope.system=all&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 226 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:41.272545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9be40-f06d-4aa6-9f63-c29a99172a27 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.273088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9be40-f06d-4aa6-9f63-c29a99172a27 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.273267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb9be40-f06d-4aa6-9f63-c29a99172a27 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.275020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 142/282] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:41.280764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.281671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.282134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.307812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:41.309359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:41.396042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 141/283] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:41.465635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9be40-f06d-4aa6-9f63-c29a99172a27 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:41.543247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:41.602876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb9be40-f06d-4aa6-9f63-c29a99172a27 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:41.647879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:41.648567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.648851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.649568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.660747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:41.665800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:41.666726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:41.667549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:41.671541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 143/284] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:41.677207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:41.704079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:41.822708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:41.844727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823a3104-b762-4e2f-816b-f28079846fb7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:41.867741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0858b04f-c6e7-4199-9639-79ec41866093 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:41.903323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:41.910843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:41.911464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.911735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.912000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.912669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:41.912967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:41.914377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:41.934607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 142/285] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:41.936653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:41.937302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:41.937584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:41.937849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:41.939190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:41.960128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:41.965012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:41.965889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:41.967041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:41.968351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 144/286] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/roles/reader => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:41.974745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.109619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7f6b88-6797-4201-87fb-e905c56a7f7a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.148022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.148829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.149106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.149372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] PATH_INFO: `/v3/users/947b64c3bee445379f566c2f1343c813` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.150551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] RBAC: Authorizing `identity:get_user(user_id=947b64c3bee445379f566c2f1343c813)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.152847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.154848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 143/287] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/users/947b64c3bee445379f566c2f1343c813 => generated 297 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:42.160844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.190947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:42.205024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9309d27-688e-4091-912e-ffaa31ffc82c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.276500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb37689-1665-4a5f-b95b-8c6fa311941a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.290597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.291216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.291496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.291773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.292441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:42.292741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.294116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.318994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.319629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.319863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.320136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.321062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 145/288] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:41 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 348 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:42.327245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.383929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:42.456988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2affc75-7277-4949-8c4b-974c6dbbb59f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.495690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.497617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.498017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.499225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.499789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.500202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] PATH_INFO: `/v3/users/6b0354746a8447a698a3203af8eb0526` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.502961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] RBAC: Authorizing `identity:get_user(user_id=6b0354746a8447a698a3203af8eb0526)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.507216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.512248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 146/289] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:42 2022] GET /identity/v3/users/6b0354746a8447a698a3203af8eb0526 => generated 297 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:42.518289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.552430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:42.692968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13090fc-b7be-4406-8201-bb2a5b71dc64 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.718436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:42.727531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 144/290] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 567 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:42.730968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.752708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.754073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.754607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.755078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.756769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.758570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.763548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 147/291] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:42 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:42.766921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:42.926907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f991eb-4360-481a-9f08-945c1ae649d4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:42.965803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:42.966970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:42.967452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:42.968337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/users/6b0354746a8447a698a3203af8eb0526/roles/248695be26f54d9d9e20d976c4890f7d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:42.970561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, user_id=6b0354746a8447a698a3203af8eb0526, role_id=248695be26f54d9d9e20d976c4890f7d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:42.972351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:42.990271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 148/292] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:05:42 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/users/6b0354746a8447a698a3203af8eb0526/roles/248695be26f54d9d9e20d976c4890f7d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:43.032783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e027cc-7dd2-4b8f-998f-b37955720873 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:43.071103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:43.091413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:43.092467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.092781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.093058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.094949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:43.096498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f7c4b9-a4f5-4dda-9445-8b00bd0b869f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:43.098610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 145/293] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:42 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:43.104593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:43.158921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:43.227118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d961d9a6-bd46-4875-bad2-acea8f6b1139 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:43.264708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:43.265718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.266197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.266687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.267622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=947b64c3bee445379f566c2f1343c813) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:43.268150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:43.269804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bdee12-c82c-4f3c-84d8-f4aef0b5aea6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:43.276805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 149/294] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:43 2022] GET /identity/v3/role_assignments?user.id=947b64c3bee445379f566c2f1343c813&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:43.346135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec9be-2a5d-4536-b121-daceabd2d987 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.346680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec9be-2a5d-4536-b121-daceabd2d987 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.346786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bec9be-2a5d-4536-b121-daceabd2d987 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.348397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 146/295] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:43.353712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274882-1f9e-48f9-a850-c89f06a0f92d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.354048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274882-1f9e-48f9-a850-c89f06a0f92d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.354272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0274882-1f9e-48f9-a850-c89f06a0f92d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.356980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4cd0-9f55-44f2-ae99-9137a9eb04bb None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.358610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4cd0-9f55-44f2-ae99-9137a9eb04bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.359142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a6b4cd0-9f55-44f2-ae99-9137a9eb04bb None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.360908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 147/296] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:43.366346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.366868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.367368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.375608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b0274882-1f9e-48f9-a850-c89f06a0f92d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:43.376845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b0274882-1f9e-48f9-a850-c89f06a0f92d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:43.410683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:43.411903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:43.490383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 150/297] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:43.491987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629bb3d-57b8-41ee-84c6-867a60691a44 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.492407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629bb3d-57b8-41ee-84c6-867a60691a44 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.492812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b629bb3d-57b8-41ee-84c6-867a60691a44 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.494175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 151/298] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:43.499892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.500759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.501205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.502952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 148/299] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:43.522009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:43.523814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:43.553199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:43.602393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 152/300] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:43.784238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:43.805256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:43.915573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c8b35f-bc4d-41f8-b3c0-9baf03144bf1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:43.923864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac505df2-9d5a-4cda-a0b8-32a654d6dcf6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:43.938042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:43.953762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:43.954527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.954886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.955152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.959354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:43.960058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:43.960262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:43.960530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:43.961638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:43.963097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:43.966980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 153/301] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:43.966980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:43.968202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:43.973741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:43.974918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:43.975917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:43.977928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 149/302] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:43.978949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.003215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:44.076554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b766835a-0af7-4e37-afb6-b33272b496ca None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.088498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5605b44e-0485-44c1-9a34-c95edca68af3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.111076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.111698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.111979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.112251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.122129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:44.125737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.126524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.126952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.127361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] PATH_INFO: `/v3/groups` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:44.127859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:44.128840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 154/303] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:44.131016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.134224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.135760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.153510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 150/304] 10.222.0.46 () {60 vars in 1134 bytes} [Tue Aug 30 04:05:43 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:05:44.153510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.226421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:44.254471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:44.267417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0ae01e-f55e-4ca7-9188-6cda3e1f165d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.318463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.319084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.319359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.319631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.320301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] RBAC: Adding query filter params (name=reader) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:44.320601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.321972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.324595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c811e087-4f2e-4f6b-a9ad-0e63b8d9673b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.332648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 155/305] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:44 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:44.338809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.365442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.366207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.366585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.366931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.367670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:44.368045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.369925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.379195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 151/306] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:44.385376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.458706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243acc34-b6c3-4268-9635-fc1f45844ccf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.500648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.501951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.502427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.502864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] PATH_INFO: `/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.518830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] RBAC: Authorizing `identity:get_user(user_id=7dd4d7b1636a4ab0954b6059fbe144b0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.519159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530bf9cd-d9ee-416d-a67e-b007a34545bd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.520666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.520917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:44.527966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 156/307] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:44 2022] GET /identity/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0 => generated 293 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:44.529680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.545662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:44.564377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.565254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.565676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.566171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] PATH_INFO: `/v3/users/fe3320dfbcde4f178c454191a9b4c94f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.567739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] RBAC: Authorizing `identity:get_user(user_id=fe3320dfbcde4f178c454191a9b4c94f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.569487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.571534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 152/308] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:44 2022] GET /identity/v3/users/fe3320dfbcde4f178c454191a9b4c94f => generated 293 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:44.578601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:44.649021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1744e7-f938-463f-ac04-daa5b501ae41 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.697009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.697651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.697929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.698198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.699235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, user.id=7dd4d7b1636a4ab0954b6059fbe144b0, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:44.699544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.701919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67028dc4-3708-4312-af4c-7e8831e6fc8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.707312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aba729f-ebdb-44d0-9ee4-d572ffdcb144 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:44.708279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 157/309] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 30 04:05:44 2022] GET /identity/v3/role_assignments?user.id=7dd4d7b1636a4ab0954b6059fbe144b0&scope.system=all&role.id=248695be26f54d9d9e20d976c4890f7d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:44.751970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:44.753006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:44.753489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:44.753965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] PATH_INFO: `/v3/system/users/fe3320dfbcde4f178c454191a9b4c94f/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:44.755889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=fe3320dfbcde4f178c454191a9b4c94f, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:44.757580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a483f73f-884b-45c9-8945-1a84ba268f57 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:44.769692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 153/310] 10.222.0.46 () {58 vars in 1334 bytes} [Tue Aug 30 04:05:44 2022] PUT /identity/v3/system/users/fe3320dfbcde4f178c454191a9b4c94f/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:45.146909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8413-ed67-4698-94a5-8d202931a1d1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:45.147284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8413-ed67-4698-94a5-8d202931a1d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:45.147746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-475a8413-ed67-4698-94a5-8d202931a1d1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:45.149048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 158/311] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:45.154960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:45.155237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:45.155497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:45.174145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:45.175370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:45.208593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:45.252095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 154/312] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:45.519663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8413-ed67-4698-94a5-8d202931a1d1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:45.598631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:45.626691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-475a8413-ed67-4698-94a5-8d202931a1d1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:45.661649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:45.662275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:45.662563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:45.662877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:45.673862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:45.678814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:45.679715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:45.680512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:45.684678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 159/313] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:45 2022] GET /identity/v3/roles/reader => generated 84 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:45.686171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:45.797556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c193765-ce89-4f99-b48f-c3f8dd8d4b16 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:45.839918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:45.840543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:45.840820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:45.841087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:45.842514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:45.842895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:45.844539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:45.853803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 155/314] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:45 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:45.860181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:45.967050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79071ffe-8663-4cdf-858e-6786863f4425 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.002369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:46.003354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.003825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.004308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] PATH_INFO: `/v3/users/6b0354746a8447a698a3203af8eb0526` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.005643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] RBAC: Authorizing `identity:get_user(user_id=6b0354746a8447a698a3203af8eb0526)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:46.007292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9727222e-fb47-4897-9e63-9cc5df8dfe9b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:46.014386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 160/315] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:45 2022] GET /identity/v3/users/6b0354746a8447a698a3203af8eb0526 => generated 297 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:46.016125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:46.045270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.124744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d091d2d5-e290-44df-ae7c-0be36f1aaf42 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.162150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:46.163135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.163611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.164101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.240129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:46.244328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:46.269960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fc85f-9023-47d0-9058-778bd4dd5348 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.270150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fc85f-9023-47d0-9058-778bd4dd5348 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.270280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fc85f-9023-47d0-9058-778bd4dd5348 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.271911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 161/316] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:46.280200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.280555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.280766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.284944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.298929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.301682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:46.302998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:46.362164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 156/317] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:46.368846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:46.386462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 162/318] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:46.471900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f4e2e0-3285-4e4f-8dea-9edda360f52f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.506596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:46.507419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.507692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.507961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.510429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.510899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:46.512341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:46.514699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 157/319] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:46 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:46.520643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:46.644175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febf0f18-04ad-4684-a093-5c034e7aa451 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.675563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.680769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:46.681380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.681659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.681930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.683542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:46.684425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, scope.project.id=2926ed17fec44a849dff42b4d98de578, user.id=6b0354746a8447a698a3203af8eb0526) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:46.684736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:46.686433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8d2a064-4451-4a89-b205-6b462bb47609 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:46.698351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 163/320] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:05:46 2022] GET /identity/v3/role_assignments?user.id=6b0354746a8447a698a3203af8eb0526&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=248695be26f54d9d9e20d976c4890f7d => generated 608 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:46.735928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2876-440f-4691-9d8b-b29f05b5a268 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.736246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2876-440f-4691-9d8b-b29f05b5a268 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.736463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603f2876-440f-4691-9d8b-b29f05b5a268 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.738862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 164/321] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:46.746910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91df5ce-477b-4299-a721-5ab26c19a7e5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.748133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91df5ce-477b-4299-a721-5ab26c19a7e5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.748781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91df5ce-477b-4299-a721-5ab26c19a7e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.769710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b91df5ce-477b-4299-a721-5ab26c19a7e5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:46.771408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b91df5ce-477b-4299-a721-5ab26c19a7e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:46.811563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4765b997-cfb2-4159-a050-ea32fece7f8e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.851904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:46.852235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.852510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.854233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.854778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:46.856346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ccccf3-2944-4c56-8bd9-acda0ca56019 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:46.856448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 165/322] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:46.857977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b031b-636d-4c47-938b-5e5ea4e21f28 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.858276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 158/323] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:05:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:46.858386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b031b-636d-4c47-938b-5e5ea4e21f28 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.858637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b031b-636d-4c47-938b-5e5ea4e21f28 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.860038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 166/324] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:46.864952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b031b-636d-4c47-938b-5e5ea4e21f28 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:46.865632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:46.866153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:46.866651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:46.889880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:46.892332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:46.975533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:46.989630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014b031b-636d-4c47-938b-5e5ea4e21f28 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:46.991849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 159/325] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:47.023054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:47.033072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.033683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.033962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.034235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] PATH_INFO: `/v3/groups` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.034888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] RBAC: Authorizing `identity:create_group()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.036308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.052512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 167/326] 10.222.0.46 () {60 vars in 1134 bytes} [Tue Aug 30 04:05:46 2022] POST /identity/v3/groups => generated 226 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:05:47.141465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.258420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88652cf7-f926-4d89-95bd-6ef980258ecd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.281500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.295016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.295656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.295935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.296206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.307641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:47.313505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:47.314482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:47.315293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 160/327] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/roles/reader => generated 84 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:47.320939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.395025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:47.424796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64616058-2977-4e4d-841b-7689cfed08db None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.463084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2e7bd3-510c-4a57-af15-f5189323127a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.468819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.469753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.470229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.470727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.483877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:47.490232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:47.491171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:47.491985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:47.496551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 168/328] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:47.497891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.506445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.507424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.507912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.508376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.509280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:47.509754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.511341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.521757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 161/329] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:47.528527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.563943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:47.632929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e31c33-ec80-4ab9-bb72-630451823ffb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.670185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e05cb0-7e12-49a0-9da4-ea16769e1db1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.680543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.681243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.681703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.681932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.683765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:47.684177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.685757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.688146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:47.698530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 169/330] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:47.707491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.720269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.720916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.721195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.721465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] PATH_INFO: `/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.722755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] RBAC: Authorizing `identity:get_user(user_id=7dd4d7b1636a4ab0954b6059fbe144b0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.724563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.730768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 162/331] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:47.732894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.836201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ba8f69-75b1-4fcf-9a69-184074b1838a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.846527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:47.871528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751b791c-1003-4ed5-9120-c7476b56a1de None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:47.878057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.878843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.879218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.879604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] PATH_INFO: `/v3/users/fe3320dfbcde4f178c454191a9b4c94f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.880845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] RBAC: Authorizing `identity:get_user(user_id=fe3320dfbcde4f178c454191a9b4c94f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.882598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.889664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 170/332] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/users/fe3320dfbcde4f178c454191a9b4c94f => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:47.891283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:47.916799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:47.917835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:47.918543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:47.918958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] PATH_INFO: `/v3/system/users/7dd4d7b1636a4ab0954b6059fbe144b0/roles/248695be26f54d9d9e20d976c4890f7d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:47.921090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=7dd4d7b1636a4ab0954b6059fbe144b0, role_id=248695be26f54d9d9e20d976c4890f7d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:47.923099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-437ba4eb-0439-48cf-99fd-3c2703babc21 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:47.934672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 163/333] 10.222.0.46 () {58 vars in 1334 bytes} [Tue Aug 30 04:05:47 2022] PUT /identity/v3/system/users/7dd4d7b1636a4ab0954b6059fbe144b0/roles/248695be26f54d9d9e20d976c4890f7d => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:48.029526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b18cc9-1050-44c0-8825-772e8bedf4ea None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:48.070499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:48.071344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:48.071760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:48.072190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:48.073068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] RBAC: Adding query filter params (role.id=65feaeff5c0046dbaf41270fb80ff8cd, user.id=fe3320dfbcde4f178c454191a9b4c94f, scope.system=all) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:48.073512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:48.075129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5859f5b0-6620-4f40-a360-fbb7c2d736be None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:48.090170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 171/334] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 30 04:05:47 2022] GET /identity/v3/role_assignments?user.id=fe3320dfbcde4f178c454191a9b4c94f&scope.system=all&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 510 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:49.526853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605fbd1-87ae-4bf8-9b17-00b82293363c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:49.527869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605fbd1-87ae-4bf8-9b17-00b82293363c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:49.528353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605fbd1-87ae-4bf8-9b17-00b82293363c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:49.530424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 164/335] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:49.535989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:49.536762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:49.537174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:49.560738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:49.562333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:49.597470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:49.619764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c19e-2274-4015-baca-0f6c931b6c01 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:49.620502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c19e-2274-4015-baca-0f6c931b6c01 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:49.620977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3e1c19e-2274-4015-baca-0f6c931b6c01 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:49.624524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 165/336] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:49 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:49.629608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ed1e-1354-413d-8383-e06ef69f2b7e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:49.629894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ed1e-1354-413d-8383-e06ef69f2b7e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:49.630429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca1ed1e-1354-413d-8383-e06ef69f2b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:49.651624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 172/337] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:49.671355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aca1ed1e-1354-413d-8383-e06ef69f2b7e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:49.672910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aca1ed1e-1354-413d-8383-e06ef69f2b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:49.740403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:49.775904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 166/338] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:50.085271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.221530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a672362-9d34-463b-b042-60a9891e2857 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:50.250487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ee2a-a48f-4b08-859b-847a534a4b72 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.251164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ee2a-a48f-4b08-859b-847a534a4b72 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.251721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a4ee2a-a48f-4b08-859b-847a534a4b72 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.253661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 167/339] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:50.264283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:50.264915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.265244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.265463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.271976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.272771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.273262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.284576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:50.289629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:50.290599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:50.291428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 173/340] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:50.292097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.307546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:50.309069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:50.339434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:50.394753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 168/341] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:50.396023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.490277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d321542-8a9a-46cc-9f8f-d94f46b09383 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:50.512722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdff216-f1ee-4142-aef4-b85286038b0f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:50.521554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:50.554116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:50.555604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.556117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.556600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.557466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:50.557970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:50.559603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:50.562548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:50.563190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.563476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.563739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.570890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 169/342] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:50.576868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.592938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:50.601093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:50.602039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:50.602908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:50.610950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 174/343] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:50.614576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.781158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:50.786006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8cf6b15-80e4-445e-a521-a0db8c438df0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:50.805841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027d3b88-20e0-45e1-9176-c6043f8bb9e6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:50.829018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:50.829754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.830281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.830759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.853989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:50.854844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:50.855284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:50.855699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:50.857062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:50.857719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:50.859300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:50.867961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 175/344] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:50.869108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:50.871001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:50.912691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:50.914300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27c363f-707d-4890-83c4-8ed960890eab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:50.984155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73bdde1-d0e0-403a-ad77-40610872ea58 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.024982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 170/345] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 04:05:51.026505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa29c-e0d5-429b-8978-2d756bad0b8a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.026934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa29c-e0d5-429b-8978-2d756bad0b8a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.027826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abfa29c-e0d5-429b-8978-2d756bad0b8a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.028343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.029937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.030442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.030548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 171/346] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:51.031051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.031257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfa29c-e0d5-429b-8978-2d756bad0b8a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.042960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:51.049247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:51.050290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:51.051143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:51.058332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 176/347] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:51.059481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.069166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.105345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.146274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abfa29c-e0d5-429b-8978-2d756bad0b8a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.181081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220ae54f-024c-4318-91ed-55c207214d1b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.183902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.184517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.185453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.185855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.226727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.227780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.228302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.228767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.230278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.232091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21129d9b-ab5d-4606-8cf8-8c63a66a7b6b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.234290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 177/348] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.236402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.237047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.237611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.264280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:51.264476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.265488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:51.265891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.336868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.346806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 178/349] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:51.347902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.358887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.383396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 172/350] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:51.384255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.493656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220595a-5cfb-493e-9b0a-28a1efdd2227 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.501219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edb6e74-e0ae-43f3-b2a4-5fd7a7e54ff2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.538278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.538278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.538278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.538278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.541848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.543354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.545612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 173/351] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.546232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.561957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.562878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.563718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.564228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.565195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:51.565656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.567211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.579849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 179/352] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.580966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.582606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.662915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1599156-bb33-402c-9ab8-25682124e434 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.670381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.704527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.705169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.705449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.705721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.706883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.708334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.710274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 174/353] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.711046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.711554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805ad987-c8d8-48c6-9f06-1edc17c9b602 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.753042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.753651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.753923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.754186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.755023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:51.755322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.756856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e852a8-5ef1-43ee-81c2-387b11df921a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.764076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 180/354] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=793acbb060a2436192eb2c7413739e83 => generated 260 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.765301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa233659-8978-4731-a83b-7c8a14b058fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.765579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa233659-8978-4731-a83b-7c8a14b058fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.765902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa233659-8978-4731-a83b-7c8a14b058fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.772179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 181/355] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:51 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:51.773383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa233659-8978-4731-a83b-7c8a14b058fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:51.821441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.824537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1cae37-170d-477e-b425-fdf4052770ee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.895619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.896648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.897130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.897859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.933466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa233659-8978-4731-a83b-7c8a14b058fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:51.979018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:51.988234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:51.989211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.989696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.990164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:51.991517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:51.993023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54530521-0fd1-4db3-b803-e82a333b4fc1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:51.998286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 182/356] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:51.998286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:51.998286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:51.998286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.003546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:52.006314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:52.017325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:52.018663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:52.108881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 183/357] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:52.109963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.154640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:52.163073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 175/358] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:52.163974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.291022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458664b-629c-4a1d-b7e7-b65a5f6e6fcd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:52.345023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef65cb5-d9e2-48ac-8d37-ad83059e269c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:52.347772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:52.348430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.348710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.348978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.351186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:52.366073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:52.371201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:52.372266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:52.373213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 184/359] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:52.373890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.390600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:52.391530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.391797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.392076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.392997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:52.393311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:52.395507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89d230b-8456-4143-a523-2cddf28c5ecd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:52.404227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 176/360] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 260 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:52.406075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0cb14-58b3-4816-8e4d-cfec737052fc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.406528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0cb14-58b3-4816-8e4d-cfec737052fc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.406807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e0cb14-58b3-4816-8e4d-cfec737052fc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.408326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 177/361] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:52.409102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0cb14-58b3-4816-8e4d-cfec737052fc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.552678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:52.562432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200018d1-b2b5-456b-af9e-9237832bd620 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:52.580059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e0cb14-58b3-4816-8e4d-cfec737052fc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:52.610725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:52.612046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.612549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.613082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.615415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:52.617162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f4f871-966f-45e5-a099-025c54df0548 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:52.619461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 185/362] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:52.620920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.621412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.621908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.634304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:52.634304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.634304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.634304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.635015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] RBAC: Adding query filter params (name=reader) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:52.635015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:52.635015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:52.644628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 178/363] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:52.645473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.655276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:52.656909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:52.731563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:52.747998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 186/364] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:52.749962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.809211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8184014d-8b10-46f2-8923-897522563e7b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:52.854010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:52.867761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:52.869379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:52.870156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:52.870750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:52.906926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:52.912409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:52.913358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:52.914175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 179/365] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:52.914901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:52.952111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d0c5a-25c6-49a2-876a-2c146d0a5370 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.005798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.006816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.008718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.009200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.011765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.013819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.016102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 187/366] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:53.016886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.074861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad2010-c665-4ee8-8865-1b02ac81915a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.103641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:53.136022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.137215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.138708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.139251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] PATH_INFO: `/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.140660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] RBAC: Authorizing `identity:get_user(user_id=7dd4d7b1636a4ab0954b6059fbe144b0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.142373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.144689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 180/367] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:05:52 2022] GET /identity/v3/users/7dd4d7b1636a4ab0954b6059fbe144b0 => generated 293 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:53.145878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.163689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d9dcc-ba97-4975-acd8-0e19fa88a222 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.201921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.202576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.202848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.203111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.203770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:53.204066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.205422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.216796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 188/368] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:53.217751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.220353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:53.292844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e11db5-d0d2-4a73-8766-0e6f7fc7174a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.343354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.344335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.344833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.345300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.346236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=2926ed17fec44a849dff42b4d98de578) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:53.346766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.348385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.358277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99b54d7-d48a-4a9b-b81c-491296acf4de None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.358640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 181/369] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 260 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:53.359793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.381919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:53.405336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.406467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.406752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.407019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.433240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:53.443372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:53.444352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 189/370] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/roles/admin => generated 83 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:53.445574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.552369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0906f9b0-8749-4887-ba8e-c6ab9bf85ebd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.595637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:53.627556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68618724-72a3-4753-8e1c-68c96d928231 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.634995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.635624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.635910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.636187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.636934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] RBAC: Adding query filter params (role.id=248695be26f54d9d9e20d976c4890f7d, user.id=7dd4d7b1636a4ab0954b6059fbe144b0, scope.system=all) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:53.637242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.639724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.659060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 182/371] 10.222.0.46 () {56 vars in 1308 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/role_assignments?user.id=7dd4d7b1636a4ab0954b6059fbe144b0&scope.system=all&role.id=248695be26f54d9d9e20d976c4890f7d => generated 510 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:53.659060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:53.691496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.692142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.692425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.692698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.785323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:53.844824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.846492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:53.896260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36823f89-6fdc-49ff-aabf-dd82a3ba2a8c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:53.976410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:53.977037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:53.977311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:53.977577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:53.978280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:53.978586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:53.980001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.001450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 183/372] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 344 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:54.007461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.075398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 190/373] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 630 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:54.079254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.081057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.186769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77dfb7c2-6071-4628-b20f-0ad1b765a0f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.229608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.230223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.230748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.231020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.267141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.301438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50792962-a74e-4d69-8adf-f8f7e5a94f90 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.351075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.352025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.352305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.352573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.353760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.355931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.360549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 191/374] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:54.371765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.396337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.398119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b35912-5bc4-4fdd-b9cd-be702b8b7851 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.452105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.511235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871e3d70-db5b-43b1-b156-782d0c491953 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.520855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 184/375] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 514 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:05:54.522505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c55e6d-a9cc-4e75-be23-1633442cd3f2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.522790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c55e6d-a9cc-4e75-be23-1633442cd3f2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.523057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c55e6d-a9cc-4e75-be23-1633442cd3f2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.524464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 185/376] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:54.526355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c55e6d-a9cc-4e75-be23-1633442cd3f2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.550855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.551495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.551774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.552144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.553357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.555037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f6ebac-5bff-4e09-ac23-e9a92c50873e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.557069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 192/377] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:54.558290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.558577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.558844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.578722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:54.579928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:54.589405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.640221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c55e6d-a9cc-4e75-be23-1633442cd3f2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.672844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 193/378] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:54.673872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.686112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.686785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.687062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.687444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] PATH_INFO: `/v3/projects/b409aaa1029446de8dc3bfb718ab67c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.688600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] RBAC: Authorizing `identity:get_project(project_id=b409aaa1029446de8dc3bfb718ab67c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.690126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.692202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 186/379] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/projects/b409aaa1029446de8dc3bfb718ab67c6 => generated 344 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:54.697235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.708892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.784778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4f9e80-b6ff-4ced-ae54-9f8ea1a98b59 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.820605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.820819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee7838-ded4-44f8-aa0a-24c4e644732f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:54.821215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.821487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.821750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.822529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=2926ed17fec44a849dff42b4d98de578) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:54.822831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.824251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.831408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 194/380] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=793acbb060a2436192eb2c7413739e83 => generated 260 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:54.862377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:54.863032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.863309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.863579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] PATH_INFO: `/v3/groups/060744a9093246598a5c923a470d3207` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.869700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] RBAC: Authorizing `identity:get_group(group_id=060744a9093246598a5c923a470d3207)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:54.871242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbaa9ef-6019-40ef-a762-06a9f4b356fd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:54.873776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 187/381] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/groups/060744a9093246598a5c923a470d3207 => generated 229 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:54.879715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:54.890115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:54.980236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b378e9-b372-4eec-a71a-6fcdd6e2e42b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.980536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b378e9-b372-4eec-a71a-6fcdd6e2e42b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.980691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b378e9-b372-4eec-a71a-6fcdd6e2e42b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:54.983026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 188/382] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:54.988989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:54.989679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:54.990181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.010229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf6d508-f494-4bd3-be9a-e513089c80a1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:55.032631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:55.034167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:55.049843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:55.050473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:55.050814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:55.051081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.051812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] RBAC: Adding query filter params (group.id=060744a9093246598a5c923a470d3207, role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=b409aaa1029446de8dc3bfb718ab67c6) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:55.052113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:55.053539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:55.060076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 195/383] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:05:54 2022] GET /identity/v3/role_assignments?group.id=060744a9093246598a5c923a470d3207&scope.project.id=b409aaa1029446de8dc3bfb718ab67c6&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 260 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:55.156497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:55.165652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 189/384] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:55.239924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.354775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:55.403892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e49b2e-ef88-43e0-b3ed-9c16c4502cad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:55.453867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:55.454701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:55.455039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:55.455363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.473180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:55.480919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:55.482733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:55.484013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:55.484639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.484714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 196/385] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:55.489722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.607092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e112f0f7-4a49-4a14-9e5a-3180fbcd9246 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:55.644708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:55.650462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:55.651224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:55.651663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:55.652580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.654161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:55.654707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:55.656371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:55.667794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 197/386] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:55.673520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.743670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae0bd2d-9b74-4655-bc37-4cf478974603 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:55.812080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea07114-068e-4de2-99ec-8d457fd1fb8c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:55.814345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:55.815012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:55.815277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:55.815548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.837258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:55.842574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:55.843448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:55.844239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:55.849462 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 190/387] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 369 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:55.850330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.883543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:55.884402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:55.884682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:55.885097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:55.893736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:55.895226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:55.897800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 198/388] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:55.904646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:55.986777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.038629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.073067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee3c41c-ddd1-44cb-98d2-90d307f30bc4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.096593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206561f-c93c-40b4-8cc4-e95df9e7fbf0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.118012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.118651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.118925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.119191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.119845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:56.120141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.121500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48b1aac5-c38a-4c03-8793-d6f7d6637403 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.133125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 191/389] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:56.134441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616027f-c8e0-49be-a643-55a35ca5af39 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.134655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616027f-c8e0-49be-a643-55a35ca5af39 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.134926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b616027f-c8e0-49be-a643-55a35ca5af39 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.136641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 192/390] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:56.138638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.138937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616027f-c8e0-49be-a643-55a35ca5af39 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:56.139840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.144331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.144846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.146418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.148313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29620aa8-54fd-484f-86e8-d240e7cd7cac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.150525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 199/391] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:55 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:56.152134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.152615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.153731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.173875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:56.175623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:56.255759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b616027f-c8e0-49be-a643-55a35ca5af39 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.257512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 200/392] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:56.258642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:56.282670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.302332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.302969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.303244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.303515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.304617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.306048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.307655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 193/393] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:56 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:05:56.314516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:56.385004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.445667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6760eb9-a6f0-474a-8392-b26b2c1e770e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.478142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f22350-f3bb-4c62-b1c5-08d346e02f95 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.481689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.482353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.482629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.482894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.484033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.485473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.487543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 194/394] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:56 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:56.493205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:56.543001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.543680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.544501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.544990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/groups/442b0c9810a34434a6fa215e298cd506/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.551004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, group_id=442b0c9810a34434a6fa215e298cd506, role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.552452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.566630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.575035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 201/395] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 30 04:05:56 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/groups/442b0c9810a34434a6fa215e298cd506/roles/793acbb060a2436192eb2c7413739e83 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:56.622943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d54581-ccd5-4d8a-9e14-ee469381d239 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.643802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:56.686394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.686394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.686394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.686394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/groups/442b0c9810a34434a6fa215e298cd506/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.689324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] RBAC: Authorizing `identity:create_grant(project_id=d7cf2cbacea7491b9f7ddad440c0d237, group_id=442b0c9810a34434a6fa215e298cd506, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:56.691704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:56.718216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 195/396] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 30 04:05:56 2022] PUT /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237/groups/442b0c9810a34434a6fa215e298cd506/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:05:56.822659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233dd81b-1a7a-4a71-868f-a6891410e538 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:56.855174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:56.882116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:56.882955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:56.883260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:56.883528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:56.899942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:56.905579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:56.906606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:56.907410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 202/397] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:05:56.913127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:57.044640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf18f91-2400-401a-bf28-c6587c2d57d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:57.064887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:57.087744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:57.088710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.089180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.089641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.090516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:57.091567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:57.093217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fedab0-ae44-4500-8a50-fe61756f991e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:57.106130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 196/398] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:05:56 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:57.113569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:57.264701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3e76fd-576f-49d0-95aa-d2e63cbc0e67 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:57.311250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:57.311887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.312166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.312435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.313677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:57.314981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f9c6-73bc-4004-9c3a-6388a03023fa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.315198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:57.315656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f9c6-73bc-4004-9c3a-6388a03023fa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.316125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d7f9c6-73bc-4004-9c3a-6388a03023fa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.316831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 203/399] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:57 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:57.318095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 197/400] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:57.322365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:57.326772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b2e5-e542-4011-bd8a-b40dfb842196 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.327322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b2e5-e542-4011-bd8a-b40dfb842196 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.327783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab25b2e5-e542-4011-bd8a-b40dfb842196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.382295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:57.383252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ab25b2e5-e542-4011-bd8a-b40dfb842196 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:57.384471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab25b2e5-e542-4011-bd8a-b40dfb842196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:57.447269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d095e7-25e9-4e74-a0ce-15ce0ffc3221 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:57.495810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:57.496525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.496804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.497818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.499869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:57.501462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:57.503611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 204/401] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:57 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:57.510308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:57.533605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 198/402] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:57.623046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5d7bd0-d331-4868-b72c-8004d0f5f0b4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:57.643048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:57.671759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:57.672420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.672697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.672967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/groups/442b0c9810a34434a6fa215e298cd506/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.675974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, group_id=442b0c9810a34434a6fa215e298cd506, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:57.677463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d49584b1-39bd-460f-8872-1e53a6a25fe7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:57.708542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 205/403] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 30 04:05:57 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/groups/442b0c9810a34434a6fa215e298cd506/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:57.757154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4284-9230-4e1f-bec0-96e2086ed97b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.757774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4284-9230-4e1f-bec0-96e2086ed97b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.758237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a4284-9230-4e1f-bec0-96e2086ed97b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.760064 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 199/404] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:57.766081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:57.766440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:57.766651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:57.789548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:57.790728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:57.874635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 206/405] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:57.920027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a4284-9230-4e1f-bec0-96e2086ed97b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.046704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a4284-9230-4e1f-bec0-96e2086ed97b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.059113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:58.088187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.089095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.089511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.089920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.107328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:58.112483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:58.113562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:58.114581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:58.116525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 200/406] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:57 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:58.122503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.263525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67c3026-d1e6-47d8-9bc8-0f0fec0983ce None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.265938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:58.287430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.302591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.303117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.303392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.303657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.304317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:58.304613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:58.305988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:58.315229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 207/407] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:58.320330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.431336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b46243-c718-4385-bd3e-25da7258f4de None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.470827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.471625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.471907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.472176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.475134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:58.476556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:58.478464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 208/408] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:58.484575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.518803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e356a2-dadf-472e-825c-0639e8cf453a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.539958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:58.598987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.599895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.600313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.600731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.603386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb27f59c-70e0-4f5b-91fa-dfd1dbc45b8f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.616042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:58.626291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:58.627476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:05:58.628524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:05:58.630888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 201/409] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:58.636203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.644242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.644947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.645294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.645633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.658486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:58.660081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:58.662173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 209/410] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:58.673512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:58.704718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:58.837448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7783479f-4637-4d09-8da3-648e0e4d4c4e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.842549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:58.853521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb61b030-48a0-4596-9ff2-af129e3e950f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:58.914841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.915517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.915793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.916065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578/groups/442b0c9810a34434a6fa215e298cd506/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.918278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] RBAC: Authorizing `identity:create_grant(project_id=2926ed17fec44a849dff42b4d98de578, group_id=442b0c9810a34434a6fa215e298cd506, role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:58.919824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:58.925028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:58.925643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:58.925922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:58.926191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:58.926878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:05:58.927175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:58.928552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:58.941091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 210/411] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 30 04:05:58 2022] PUT /identity/v3/projects/2926ed17fec44a849dff42b4d98de578/groups/442b0c9810a34434a6fa215e298cd506/roles/793acbb060a2436192eb2c7413739e83 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:58.950603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 202/412] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:58.956268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:59.109781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b37fa-2c08-4a4c-a116-270682bc3497 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:59.154625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:59.155647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.156131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.156612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] PATH_INFO: `/v3/groups/060744a9093246598a5c923a470d3207` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.157943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] RBAC: Authorizing `identity:get_group(group_id=060744a9093246598a5c923a470d3207)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:59.159601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e0dc35-b0ce-4953-9410-7d60a135a329 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:59.161726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 211/413] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:05:58 2022] GET /identity/v3/groups/060744a9093246598a5c923a470d3207 => generated 229 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:59.167721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:59.194672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30826-e234-4b59-91c4-6abb01b0ee34 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.195266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30826-e234-4b59-91c4-6abb01b0ee34 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.195780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c30826-e234-4b59-91c4-6abb01b0ee34 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.197639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 212/414] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:05:59.203365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7591d73-fe90-4f4b-ac1d-8d4b7e6361ec None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.203938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7591d73-fe90-4f4b-ac1d-8d4b7e6361ec None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.204472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7591d73-fe90-4f4b-ac1d-8d4b7e6361ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.229331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c7591d73-fe90-4f4b-ac1d-8d4b7e6361ec None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:59.231178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c7591d73-fe90-4f4b-ac1d-8d4b7e6361ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:59.273272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:59.277621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:59.281677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda97849-3c61-405a-81c1-55afa3ad67b9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:59.320856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:59.321593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.321941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.322306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] PATH_INFO: `/v3/projects/b409aaa1029446de8dc3bfb718ab67c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.323658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] RBAC: Authorizing `identity:get_project(project_id=b409aaa1029446de8dc3bfb718ab67c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:59.325780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:59.328075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 203/415] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:05:59 2022] GET /identity/v3/projects/b409aaa1029446de8dc3bfb718ab67c6 => generated 344 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:05:59.334469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:59.447608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f0c393-4e9d-47c9-8265-6758ca9516f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:59.455977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 213/416] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:05:59.484368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:59.485252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.485633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.486007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] PATH_INFO: `/v3/projects/b409aaa1029446de8dc3bfb718ab67c6/groups/060744a9093246598a5c923a470d3207/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.488088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] RBAC: Authorizing `identity:create_grant(project_id=b409aaa1029446de8dc3bfb718ab67c6, group_id=060744a9093246598a5c923a470d3207, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:05:59.489616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aff31a-bf1d-4eb9-819e-9c987ce2f98d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:05:59.510360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 204/417] 10.222.0.46 () {58 vars in 1442 bytes} [Tue Aug 30 04:05:59 2022] PUT /identity/v3/projects/b409aaa1029446de8dc3bfb718ab67c6/groups/060744a9093246598a5c923a470d3207/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:05:59.558020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a802f4-e351-489d-83f2-2fe82c06aacb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.558690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a802f4-e351-489d-83f2-2fe82c06aacb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.559126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a802f4-e351-489d-83f2-2fe82c06aacb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.561416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 214/418] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:05:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:05:59.569900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558358-906e-40c7-a667-cc40f82a767d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.570351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558358-906e-40c7-a667-cc40f82a767d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.570723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66558358-906e-40c7-a667-cc40f82a767d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.599561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66558358-906e-40c7-a667-cc40f82a767d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:05:59.600952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66558358-906e-40c7-a667-cc40f82a767d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:05:59.623546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:05:59.698535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 205/419] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:05:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:05:59.771271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a802f4-e351-489d-83f2-2fe82c06aacb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:05:59.897865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a802f4-e351-489d-83f2-2fe82c06aacb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:05:59.948867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:05:59.950347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:05:59.950928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:05:59.951398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:05:59.969118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:05:59.974238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:05:59.975144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:05:59.975960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:05:59.978519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 215/420] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:05:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:05:59.987997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66558358-906e-40c7-a667-cc40f82a767d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.052072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.087969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.171989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66558358-906e-40c7-a667-cc40f82a767d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.221799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.222351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.222626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.222891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.224039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:00.224470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:00.226056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:00.235595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 206/421] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:05:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:00.240997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.244026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922e2cdb-1fda-46c3-af28-e80080542d73 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.275960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.321185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.322201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.322733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.323204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.337905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5ed89972-6e88-48fb-8bda-4a869d85b09f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:00.344304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:00.345178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:00.345990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 216/422] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:00.351624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be4bf0-3f58-43a0-8123-710b55c3507e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.351624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be4bf0-3f58-43a0-8123-710b55c3507e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.351624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1be4bf0-3f58-43a0-8123-710b55c3507e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.351624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 217/423] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:00.352384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be4bf0-3f58-43a0-8123-710b55c3507e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.391112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2d9339-573b-4853-8904-9c0acfd97e35 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.437794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.438434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.438736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.439532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.454963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.478738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1be4bf0-3f58-43a0-8123-710b55c3507e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.517109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.518164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.518795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.519253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.520141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] RBAC: Adding query filter params (name=member) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:00.520737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:00.522767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14c3bdac-84c6-4194-a1c0-46fa0e3aaad2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:00.526054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:00.527572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:00.537654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 218/424] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:00.539347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.539865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.540434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.542660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.562399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:00.563925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:00.643305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 207/425] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:00.644753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.650487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 219/426] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:00.651327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.803322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.828846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28207778-12e5-439a-8aed-181ccdf03290 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.844358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:00.869834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.871126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.871566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.872679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.885009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e44ea0-3d2c-4d2b-9e95-5e061834de7f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:00.945234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:00.946064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:00.946465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:00.946880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:00.948154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:00.949781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:00.952532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 220/427] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:00.953399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:00.958215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:00.959844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.074669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53424b55-7dd2-4e23-b2e9-b6926507163b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.088019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 208/428] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:01.093842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.110755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.117630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.118280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.118558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.118825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.127426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.131810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:01.138893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:01.139811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:01.140610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 221/429] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:01.142301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.209433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02cb742-19a2-4af3-9b55-7918207fee48 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.243582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17083158-26e0-4a98-9603-797633896708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.244209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083158-26e0-4a98-9603-797633896708 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.244490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083158-26e0-4a98-9603-797633896708 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.244761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17083158-26e0-4a98-9603-797633896708 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.245868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17083158-26e0-4a98-9603-797633896708 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.247929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17083158-26e0-4a98-9603-797633896708 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.249669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 209/430] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:01.250689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab5ec-73ca-4233-8a54-b1ca63f4ac03 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.250975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab5ec-73ca-4233-8a54-b1ca63f4ac03 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.251244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11cab5ec-73ca-4233-8a54-b1ca63f4ac03 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.252330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ee6a10-716d-4a06-a012-801b3dcde7d7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.256718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 210/431] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:01.256718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab5ec-73ca-4233-8a54-b1ca63f4ac03 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.294463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.295088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.295371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.295642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] PATH_INFO: `/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.296792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] RBAC: Authorizing `identity:get_project(project_id=d7cf2cbacea7491b9f7ddad440c0d237)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.298225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a14f8b-1432-4469-bd31-9c4798e0491d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.299873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 222/432] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/projects/d7cf2cbacea7491b9f7ddad440c0d237 => generated 298 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:01.300967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.301243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.301509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.319366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:01.320546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:01.346660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.373438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11cab5ec-73ca-4233-8a54-b1ca63f4ac03 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.389081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.406279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 223/433] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:01.406279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.424345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.425341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.425620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.426164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.426882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] RBAC: Adding query filter params (name=member) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:01.427412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.429395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.441348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 211/434] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:01.442195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.508444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbff51a-31c7-49a1-92d0-4b038f25e792 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.546736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.547403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.547683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.548041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.548848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:01.549211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.550676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.556872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 224/435] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=793acbb060a2436192eb2c7413739e83 => generated 611 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:01.557677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.585070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51889e3e-ba1f-4218-babb-38c399744609 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.662359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.663352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.663822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.664303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.665685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.667385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.669802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 212/436] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:01.676015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.690811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.692841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:01.768401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b45737-4759-4c99-9b35-ca15ddd812d3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.814881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0f3aa4-b53e-49b2-98cc-e2abd8eb75e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:01.829619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.830628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.831110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.831479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.853680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:01.854421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:01.854757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:01.855093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:01.855904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:01.856777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.858208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.866397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 213/437] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=d7cf2cbacea7491b9f7ddad440c0d237&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:01.867218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:01.933651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:01.936283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470dae68-07ae-4df3-ada3-7aafff3fefb8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:01.972703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.003632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79afdfc2-fe0e-4cfd-92ee-713719ee75f2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.007363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.043599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.044285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.044671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.045132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.056998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:02.057989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 225/438] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:02.059105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f7876-abb4-47b2-b9a8-f6907e35dce6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.059444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f7876-abb4-47b2-b9a8-f6907e35dce6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.059712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161f7876-abb4-47b2-b9a8-f6907e35dce6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.061465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 226/439] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0b3cf2ff-7a17-4324-81c4-c996b946dd3d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:02.064302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:02.065400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 214/440] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:02.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.067693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.067693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.070600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f7876-abb4-47b2-b9a8-f6907e35dce6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.087763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:02.088953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:02.169854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 215/441] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:02.171384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.193103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161f7876-abb4-47b2-b9a8-f6907e35dce6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.238475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.239239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.239492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.239766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.240957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.242626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.244398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 227/442] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:02.250592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.252169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.304416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.322852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2355af68-fdcb-4bbf-b654-ca4a8c328321 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.372587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.373201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.373527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.373798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.374522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:02.374827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.376337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.390921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 216/443] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:02.393090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.397642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab48b4-5d17-44fc-ab3d-8db0e6f94daa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.444544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.445209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.445553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.445891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.447047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.448601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.455024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 228/444] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:02.456719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.522174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f0549-ace1-4a3b-b291-00d8a6d8a075 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.548954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.558495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.559574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.559859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.560125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.567089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.571126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086baa58-fa38-44fa-b556-e816c96beb67 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.605913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.606577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.606838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.607106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.607817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=65feaeff5c0046dbaf41270fb80ff8cd, scope.project.id=2926ed17fec44a849dff42b4d98de578) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:02.608118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.609519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.615410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 229/445] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=65feaeff5c0046dbaf41270fb80ff8cd => generated 611 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:02.616176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.633678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.635220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.751233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cc638f-99e1-4376-b9d6-aedb371a3f20 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.761741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 217/446] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:02.767990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.790844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.791042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.791726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.792006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.798552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.805300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:02.811413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:02.812362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:02.813164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 230/447] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:02.819816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.845433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:02.896579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc76cd89-5186-497f-a284-581e8d49f129 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.938750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509cf45-aaab-41e7-8f52-e96ca0f519b8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:02.945197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.945835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.946114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.946430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] PATH_INFO: `/v3/projects/2926ed17fec44a849dff42b4d98de578` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.947575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] RBAC: Authorizing `identity:get_project(project_id=2926ed17fec44a849dff42b4d98de578)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.949122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.951295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 218/448] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/projects/2926ed17fec44a849dff42b4d98de578 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:02.956980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:02.979241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:02.979889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:02.980011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:02.980286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:02.981997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:02.982554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:02.983960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:02.994184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 231/449] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:02.999666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:03.061507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:03.068521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef3f34c-672b-448c-980d-e0899c7c4765 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.082660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:03.099439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc53a27-9760-4658-9efc-c4c11cb6059b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.102833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.103464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.103741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.104010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] PATH_INFO: `/v3/groups/442b0c9810a34434a6fa215e298cd506` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.105124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] RBAC: Authorizing `identity:get_group(group_id=442b0c9810a34434a6fa215e298cd506)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.107262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.108839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 219/450] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/groups/442b0c9810a34434a6fa215e298cd506 => generated 226 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:03.114662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:03.134278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.134888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.135167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.135438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.204517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.206086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.214950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce4573fd-ad3e-4c69-b4d7-cdc579544432 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.249413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.250118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.250423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.250693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.251409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] RBAC: Adding query filter params (group.id=442b0c9810a34434a6fa215e298cd506, role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=2926ed17fec44a849dff42b4d98de578) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:03.251713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.253140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.258984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 220/451] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:06:03 2022] GET /identity/v3/role_assignments?group.id=442b0c9810a34434a6fa215e298cd506&scope.project.id=2926ed17fec44a849dff42b4d98de578&role.id=793acbb060a2436192eb2c7413739e83 => generated 611 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:03.290945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:03.308757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 232/452] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 04:06:03.315035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:03.347393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:03.423402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad612037-a861-416e-8f20-2fd09c6be397 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.459334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.459968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.460249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.460518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] PATH_INFO: `/v3/projects/b409aaa1029446de8dc3bfb718ab67c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.461635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] RBAC: Authorizing `identity:get_project(project_id=b409aaa1029446de8dc3bfb718ab67c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.463105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.464724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 221/453] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:06:03 2022] GET /identity/v3/projects/b409aaa1029446de8dc3bfb718ab67c6 => generated 344 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:03.471133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:03.575839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10aa1164-1df2-435f-b60d-4b980245df5c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.613245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.613872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.614143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.614470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] PATH_INFO: `/v3/groups/060744a9093246598a5c923a470d3207` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.615557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] RBAC: Authorizing `identity:get_group(group_id=060744a9093246598a5c923a470d3207)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.616979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c802e8a7-017b-4387-9eb3-728fc9e15e70 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.618591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 233/454] 10.222.0.46 () {56 vars in 1180 bytes} [Tue Aug 30 04:06:03 2022] GET /identity/v3/groups/060744a9093246598a5c923a470d3207 => generated 229 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:03.625244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:03.719076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:03.725981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d388eb79-244c-443e-90dc-92053afb15d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:03.758656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:03.759246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:03.759519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:03.759782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:03.760504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] RBAC: Adding query filter params (group.id=060744a9093246598a5c923a470d3207, role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=b409aaa1029446de8dc3bfb718ab67c6) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:03.760790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:03.762197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec9404b-c786-4e50-a1a6-80c2b1e25928 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:03.767922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 222/455] 10.222.0.46 () {56 vars in 1376 bytes} [Tue Aug 30 04:06:03 2022] GET /identity/v3/role_assignments?group.id=060744a9093246598a5c923a470d3207&scope.project.id=b409aaa1029446de8dc3bfb718ab67c6&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:06.208226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2db2-e748-4777-939b-5fd5102b1753 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.208923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2db2-e748-4777-939b-5fd5102b1753 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.208923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19d2db2-e748-4777-939b-5fd5102b1753 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.210378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 234/456] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:06.215242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e200a1-9c95-4735-9829-ce28cc3ed462 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.216143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e200a1-9c95-4735-9829-ce28cc3ed462 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.216693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e200a1-9c95-4735-9829-ce28cc3ed462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.238300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-93e200a1-9c95-4735-9829-ce28cc3ed462 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:06.239827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93e200a1-9c95-4735-9829-ce28cc3ed462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:06.336714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 223/457] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:06.411568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ee8b-13f3-4b9d-bd9b-203d017c24cd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.412189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ee8b-13f3-4b9d-bd9b-203d017c24cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.412656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa39ee8b-13f3-4b9d-bd9b-203d017c24cd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.414432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 235/458] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:06.420664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881843b6-9f3e-4733-b9b0-0b5441d7aa39 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.421208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881843b6-9f3e-4733-b9b0-0b5441d7aa39 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.421679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881843b6-9f3e-4733-b9b0-0b5441d7aa39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.447581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-881843b6-9f3e-4733-b9b0-0b5441d7aa39 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:06.449118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-881843b6-9f3e-4733-b9b0-0b5441d7aa39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:06.548458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 224/459] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:06.717566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39ee8b-13f3-4b9d-bd9b-203d017c24cd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:06.742596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e512-c9e7-4591-8a9e-48202fdf32c5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.743228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e512-c9e7-4591-8a9e-48202fdf32c5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.743699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f3e512-c9e7-4591-8a9e-48202fdf32c5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.745522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 225/460] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:06.757933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.758216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.758524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.788264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:06.795402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:06.796625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:06.828048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:06.858994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa39ee8b-13f3-4b9d-bd9b-203d017c24cd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:06.913840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:06.914498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.914778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.915046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.915688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 226/461] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:06.919986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:06.927204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:06.928502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-20fd87c1-6d42-4507-b116-921348531069 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:06.944394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:06.945331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:06.946833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:06.946833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:06.946833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:06.946833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 236/462] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:06.947060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379c4c-22d8-4e68-b090-caba67474c59 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:06.947274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379c4c-22d8-4e68-b090-caba67474c59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:06.947543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98379c4c-22d8-4e68-b090-caba67474c59 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:06.950292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 237/463] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:06.953992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98379c4c-22d8-4e68-b090-caba67474c59 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.060057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b7bb7b-2ac8-4b72-8cbf-e49ea15ab5d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.068915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98379c4c-22d8-4e68-b090-caba67474c59 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.071267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.099010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.100086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.100563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.101031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.109192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.109801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.110081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.110382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.111023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:07.111328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.112707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-189fdb97-c7ee-4d97-80e6-22cb7e404123 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.114372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.116058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.125503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.125917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 238/464] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:07.128066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.128066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.128066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:07.148954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:07.150077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:07.151165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:07.151165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.151165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:07.151165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 227/465] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:07.151165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.165801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:07.167870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:07.251288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 239/466] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:07.254939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.283940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598ec878-8558-40f1-9979-0f47215c911b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.320196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.320958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.321333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.321698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.322613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.324381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.359062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 228/467] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:07 2022] POST /identity/v3/users => generated 246 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:07.360218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.361525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.370990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a97bb2e-5e8a-45ef-adec-c320194741f2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.421828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.422723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.423152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.423592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.424410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:07.424864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.426516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.451325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.455406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 240/468] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:07.463135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.506479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ec15a7d-0db3-4f28-a2c3-4c8eece3d4f5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.543945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.544747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.545084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.545441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.557408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.559615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:07.577549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:07.578538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:07.579613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:07.579613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.579613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:07.579613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 229/469] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:07.585212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.641687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb7774e-ebe2-446f-8b55-3c51d6380df7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.691545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.692159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.692442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.692713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.693372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.694845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.718553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 241/470] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:07 2022] POST /identity/v3/users => generated 248 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:06:07.719390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.721111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d737e98-2eed-4e5c-8d02-9e04866fbb19 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.738328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.738529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:07.766482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.767450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.767832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.768206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.784547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.786034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-63249c25-934d-47a4-9882-449f7252fa4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:07.803461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:07.804471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:07.805336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:07.805336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:07.805336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:07.811268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 230/471] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:07.812739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0958-c6b5-4c7b-b597-6a8e72b22f39 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.813363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0958-c6b5-4c7b-b597-6a8e72b22f39 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.813363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783b0958-c6b5-4c7b-b597-6a8e72b22f39 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.820326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 231/472] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:07 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:07.821503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b0958-c6b5-4c7b-b597-6a8e72b22f39 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:07.872357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d63390-fff5-4b16-8b62-2de7d320ce52 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.914470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:07.914873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.915151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.915422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.916071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:07.916373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:07.917743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bab6e3fc-bd60-4377-8ae2-64aa2e03926e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:07.932423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 242/473] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:07.934723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:07.935269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:07.935875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:07.957840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783b0958-c6b5-4c7b-b597-6a8e72b22f39 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:07.959578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:07.962088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:08.002685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:08.003501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:08.003884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:08.004257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:08.005014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:08.005418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:08.006930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:08.024520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 232/474] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:08.025846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:08.038203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:08.046943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 243/475] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:08.047882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:08.056822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:08.169783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105b54ed-5071-4e40-bec4-fae3c408e4bf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:08.211424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:08.212531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:08.213014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:08.213484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:08.214378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:08.217564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:08.256875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 233/476] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:08 2022] POST /identity/v3/users => generated 248 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:08.270240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76f014-0fc0-49fe-9015-b1e74071145d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:08.314021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:08.314654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:08.314930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:08.315208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:08.315846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:08.317263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:08.355309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 244/477] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:08 2022] POST /identity/v3/users => generated 249 bytes in 308 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:08.410441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:08.528632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:08.555527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58fec0e3-4610-40e4-af6e-ee4b13864423 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:08.592985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:08.593595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:08.593911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:08.594132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:08.594835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] RBAC: Authorizing `identity:create_role()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:08.596248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:08.605285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 04:06:08.605285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 04:06:08.605285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [parameters: {'id': '039767d2f1164dfba2c0ef487854ba5b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 04:06:08.605285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 30 04:06:08.624630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return ref.to_dict() Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:06:08.625564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 04:06:08.626522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 04:06:08.627428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:06:08.629002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:06:08.631298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'id': '039767d2f1164dfba2c0ef487854ba5b', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:08.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Aug 30 04:06:08.635300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:08.636171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 234/478] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 30 04:06:08 2022] POST /identity/v3/roles => generated 144 bytes in 220 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:08.636248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:08.706591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:08.756846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d102e1-9866-41c2-a7a2-0fabe9de9f07 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:08.797144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:08.797873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:08.798278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:08.798653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:08.811907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-423cc544-a0a5-47ab-91e5-7a175c1c287e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:08.816860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:08.817979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:08.818884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 245/479] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:08.825998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:08.997600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537f573b-f4d2-4307-97c6-e481e68c42cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:09.040062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:09.040678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:09.040955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:09.041222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:09.041890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:09.042191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:09.043649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-393b67ed-6ad1-4e7a-910b-3e0df3ef599e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:09.056957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 235/480] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:09.793142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786c31b-b1af-4da5-86cf-3f102bcc6a91 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:09.794776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786c31b-b1af-4da5-86cf-3f102bcc6a91 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:09.795187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d786c31b-b1af-4da5-86cf-3f102bcc6a91 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:09.797109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 246/481] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:09.804434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5797-b212-4145-8de6-43ad85250e06 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:09.804962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5797-b212-4145-8de6-43ad85250e06 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:09.804962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74a5797-b212-4145-8de6-43ad85250e06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:09.814304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:09.838572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c74a5797-b212-4145-8de6-43ad85250e06 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:09.839768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c74a5797-b212-4145-8de6-43ad85250e06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:09.935371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 236/482] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:10.147632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b0af5-b7d7-4c39-b730-06d989b1be68 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.147975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b0af5-b7d7-4c39-b730-06d989b1be68 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.148185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b0af5-b7d7-4c39-b730-06d989b1be68 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.149847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 247/483] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:10.155933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.156221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.156499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.181716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:10.182941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:10.265251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 237/484] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:10.295218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b0af5-b7d7-4c39-b730-06d989b1be68 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:10.424018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b0af5-b7d7-4c39-b730-06d989b1be68 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:10.445494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:10.469873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:10.470540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.470824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.471098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.485252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:10.490159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:10.491058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:10.492477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 248/485] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:10.497119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:10.563308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:10.587049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:10.663480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bfa64c-8d3a-4b27-9761-9cdcb5a88df5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:10.714444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36382729-519c-4018-af17-5c28b5df2e2f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:10.723369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:10.723982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.724259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.724528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.725192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:10.725492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:10.726895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:10.740731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 238/486] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:10.741553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:10.779133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:10.780130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.780599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.781150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.800349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cf890a06-09ac-4666-807b-c26c68ac3cf4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:10.806808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:10.807675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:10.808482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:10.817634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 249/487] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:10 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:10.822830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2acd56-ca89-4caf-9c98-0a6cfd36ec5f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.823448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2acd56-ca89-4caf-9c98-0a6cfd36ec5f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.823935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b2acd56-ca89-4caf-9c98-0a6cfd36ec5f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.826863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 250/488] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:10 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:10.828534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2acd56-ca89-4caf-9c98-0a6cfd36ec5f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:10.858443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec0ad1c-3f49-47e8-8578-769e8b9dd647 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:10.872813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:10.884955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:10.897850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:10.898538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.898812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.899080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.911532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:10.913081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6ec184f1-051d-4b0a-a3a0-14b712ba667e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:10.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:10.929647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:10.930935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:10.930935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:10.930935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:10.930935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 239/489] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:10.935539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.935677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.935898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.954856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:10.956043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:10.956556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b2acd56-ca89-4caf-9c98-0a6cfd36ec5f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:10.992739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c0d516-53f8-4add-934f-966236370185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:10.993261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d516-53f8-4add-934f-966236370185 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:10.993535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d516-53f8-4add-934f-966236370185 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:10.993800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c0d516-53f8-4add-934f-966236370185 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:10.994484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0d516-53f8-4add-934f-966236370185 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:10.994834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0d516-53f8-4add-934f-966236370185 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:10.996207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c0d516-53f8-4add-934f-966236370185 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.004646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 251/490] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:10 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:11.005942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eef3-c1b3-4e6c-b179-dee936b3728d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.006221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eef3-c1b3-4e6c-b179-dee936b3728d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.006515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925eef3-c1b3-4e6c-b179-dee936b3728d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.011546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 252/491] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:11.011546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925eef3-c1b3-4e6c-b179-dee936b3728d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.036932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 240/492] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:11.038069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.124165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.129880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925eef3-c1b3-4e6c-b179-dee936b3728d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.180519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c20b35-f63c-436d-a08c-9f57862bf294 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.180712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.181387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.181673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.182200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.183793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:11.184346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.185834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fd783f4-7124-4d04-a270-1c6474d6c590 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.192165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.206075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 253/493] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:11.207408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.207693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.207962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.227298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:11.228678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:11.230552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.233458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.233959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.234457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.249415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.251830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:11.273960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:11.275023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:11.275896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:11.275896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:11.275896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:11.279748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 241/494] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:11.280834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.315522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 254/495] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:11.315522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.410625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-912150e7-e733-4d05-baaf-5342fdbd08c5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.441286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.448735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.450356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.450642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.451174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.456411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfbf05d-61d0-4669-bd0b-208dabd337ed None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.460873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.462338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:11.478666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:11.479618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:11.480549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:11.483570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 242/496] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:11.483570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.499169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.499835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.500059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.500331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.500988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:11.501288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.502708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.504445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.522464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 255/497] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:11.523389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.596818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f48c890-c0e6-4419-8f25-961cc0913031 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.632116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898a454-87f4-4921-824c-364cce02577b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.632731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cfef2d-0c5c-4cf6-93f7-710f86a6d9d0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.633176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a454-87f4-4921-824c-364cce02577b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.633591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a454-87f4-4921-824c-364cce02577b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.633993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a454-87f4-4921-824c-364cce02577b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.645522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2898a454-87f4-4921-824c-364cce02577b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2898a454-87f4-4921-824c-364cce02577b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:11.651336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:11.652286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:11.653776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 243/498] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:11.654900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898a454-87f4-4921-824c-364cce02577b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.672479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.673165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.673446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.673713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.674474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:11.674776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.676189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.722803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 256/499] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:11.722803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.728055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.749733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.774617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2898a454-87f4-4921-824c-364cce02577b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.808564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.809404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.809687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.809958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.818743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.820208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.821596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b7056b-2f90-48ed-a798-ce4b41bddc8e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:11.833992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:11.835131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 244/500] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/users/glance => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:11.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.860941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.861568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.861847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.862118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.873347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:11.879394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:11.880287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:11.881101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:11.881805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 257/501] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:11.882347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:11.938747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38acf76-56e0-42f3-83c5-429f32fb85f4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.949290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.979335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05325516-5950-4355-af93-d117f0493d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:11.979933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05325516-5950-4355-af93-d117f0493d0e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.980205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05325516-5950-4355-af93-d117f0493d0e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.980475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05325516-5950-4355-af93-d117f0493d0e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.981820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05325516-5950-4355-af93-d117f0493d0e None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:11.982117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05325516-5950-4355-af93-d117f0493d0e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:11.983509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05325516-5950-4355-af93-d117f0493d0e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:11.985900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16550a15-6855-4f8a-a0ae-d801395319fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:11.988455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:11.992578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 245/502] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:11.993745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33ebc-fd4f-4e70-8313-39d099992f8e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:11.994017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33ebc-fd4f-4e70-8313-39d099992f8e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:11.994364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd33ebc-fd4f-4e70-8313-39d099992f8e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:11.995496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 246/503] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:11 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:11.996176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd33ebc-fd4f-4e70-8313-39d099992f8e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.024515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.025175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.025396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.025669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.092241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.093686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.096094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd33ebc-fd4f-4e70-8313-39d099992f8e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.136444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.137633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.138036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.138518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.139449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:12.139754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.141201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.166017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.178590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 247/504] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:12.179421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.204919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 258/505] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:12.205765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.243381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.287271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6b20e6-cbad-46e2-ba84-428025aab43f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.312781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45745a0-1367-4812-bd89-8bf4bd0ddfd9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.322047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.322681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.322957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.323225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.323869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:12.324219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.325752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52024757-3c61-4fb0-bca5-6c2d1ff637c5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.333595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 248/506] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:12.334626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.334907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.335175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.349757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.350414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.350692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.350959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.352599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:12.353788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:12.361473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.362956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:12.378426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:12.379352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:12.380206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:12.380206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.380206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:12.380206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 259/507] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:12.380423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.402744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.432312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 249/508] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:12.433164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.493572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.507583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e896b3-aad6-472b-8704-dd4287b97087 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.556477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.556665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21465def-6a18-41d6-ab2d-be27fce77dd9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.557478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.557960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.558411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.599397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.600033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.600310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.600582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.612377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.613864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:12.630368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:12.631370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:12.632276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:12.632276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.632276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:12.632276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 250/509] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:12.632276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.646394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.648377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.671133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.749143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d94817f-e844-4558-9f9a-45843bb909ff None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.750760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.770297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 260/510] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:12.771589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.790646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.791285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.791561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.791837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.803818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.805286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:12.820576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:12.821597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:12.822506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:12.822506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:12.822506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:12.822506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 251/511] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:12.822506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.880497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c443efa-9492-41ee-ac1a-56482ac90382 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.901413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:12.920767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.921701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.921701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.921904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.922501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:12.922808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.924182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.936412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 261/512] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:12.937260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:12.941691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0624a9-c8a0-4dd4-8993-237f619fa0ae None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:12.975838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:12.976450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:12.976728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:12.976996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:12.977638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:12.977943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:12.979358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:12.996853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 252/513] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:12.996853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.010381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.038538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c95685b3-1d15-4ae0-8817-5bd7f740bafe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.071085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.071622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.071906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.072170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.083662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.085117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:13.100228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.106478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:13.107561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:13.107561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:13.107561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:13.107561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 262/514] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.107779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d284ef-4c1d-41c7-8f87-a6634c73c45c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.107859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.135748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.136649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.137386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.137713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.137952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.148281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.149727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:13.164656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.165630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:13.166554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:13.166554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:13.166554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:13.166554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 253/515] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.166963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.213771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e027f56-dbe7-400d-b407-d50b09d358ba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.235468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.248387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.249000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.249280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.249550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.250191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:13.250523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.251899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.264075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 263/516] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:13.264873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.276801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79613917-9d64-4573-b419-5b2e0a32cc14 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.311689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.312316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.312589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.312859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.320295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.321893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:13.334154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 254/517] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.335998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.362062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.372812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e62582-230a-4751-9f4c-75c3de08cbee None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.406972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.407591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.407873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.408144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.418576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.420186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:13.432117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.433009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 264/518] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.434758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.444963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a29e9af-bbb0-493c-a7fb-6eb8f437defc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.470826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.479720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.480320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.480631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.481032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.481862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:13.482169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.483574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.495458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 255/519] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:13.496783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.540001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-303ed7d4-8b56-48d1-9e06-9ac3d0eff249 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.576230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.576715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.577001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.577269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.577904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:13.578205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.579593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.591168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 265/520] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:13.591964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.596125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4cf46d-04b6-4f6e-ba8a-55e09bd1ae69 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.603540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.631498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.632144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.632417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.632686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.640393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.641858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:13.654075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.655026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 256/521] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.655973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.693447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d221b951-67b6-4e97-8d1a-ca4aec2378dd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.702120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.729292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.729889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.730172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.730470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.731830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:13.732132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.733555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.750626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 266/522] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:13.751528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.759472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a8aaa6-fa25-4c54-ab2c-fd191743f49d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.794078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.794700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.794978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.795246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.795935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:13.796235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.797664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.816079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 257/523] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:13.816922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.829132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.862487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081c2750-9cc6-40e4-a13f-b95febe544f9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.895537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.896149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.896388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.896914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.897551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.899007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.918409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb9606c-de84-4789-abdc-223d949cc4eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:13.920089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 267/524] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:13 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:13.920893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:13.927037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:13.959382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:13.960177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:13.960343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:13.960616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:13.970012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:13.972026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:13.986427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:13.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:13.988345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 258/525] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:13.988808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.045717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-124d8186-c971-4e2f-8819-c6fd3c63cc40 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.072091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.086545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.087155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.087433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.087698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.088423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:14.088766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.090533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.111710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 268/526] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:14.112161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6d4661-a66e-4fbb-a71b-4f49016524cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.112589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.151685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.153269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.153720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.154148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.190589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.229991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c955eb0e-03b6-4599-b3f3-f2f40fec0454 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.232782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.234898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.277412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.278567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.278840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.279131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.279820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:14.280121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.281530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.298416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 269/527] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:14.298416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.373387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.376240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 259/528] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:14.377962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.463656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b4006c-dd2e-44b3-94e3-d0df86362b47 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.465923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.514964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.515608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.515884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.516154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.516853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:14.517155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.518725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.525128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ebf558-033f-48fe-a523-cc588e323853 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.541857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 270/529] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:14.542686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.577235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.577235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.577235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.577235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.676151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.677913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef57b1a-394e-4466-8b3c-a2a5b6984db2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.678777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.688527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.727525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.728408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.728834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.729247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.748497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.750276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.766962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:14.772823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:14.773950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:14.775047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:14.775047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:14.775047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:14.781712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 271/530] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:14.783137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.830512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 260/531] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:14.830991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:14.927473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26952ed7-2e21-4f71-a7cd-ad872021e53f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.957075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7645a66c-e5df-4c83-9b4e-d86967e0c53f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:14.968076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:14.968836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:14.969121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:14.969395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:14.970138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=288a7a6690d3480db235d2f594bc0acd) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:14.970476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:14.971932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:14.982313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 272/532] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/role_assignments?user.id=288a7a6690d3480db235d2f594bc0acd&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:14.982313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.033279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.033963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.034606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.034885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.035629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:15.036107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.037679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.045350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.075123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 261/533] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:15.077141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.162648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.212573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7b1e037-dd2b-4a07-b453-083c40448ed9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.261080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.262468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.262596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.262830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.265866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318e15-1afb-43fe-a23e-9b9dab6a7424 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.275734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.277366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:15.292921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:15.294834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:15.295697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:15.295697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:15.295697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:15.299288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 273/534] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:15.300558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.319084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.320071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.320548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.321011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.333904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.343603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:15.365949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:15.366889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:15.367856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:15.370128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 262/535] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:15.377518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.384654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.413523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e083b88-c3fb-4377-bd5b-f53cad0b696f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.454996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.454996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.454996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.454996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.455403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:15.455403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.455403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.471825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 274/536] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:15.472988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.583314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b1b20f1-2304-4afe-8add-01f9d7b7767c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.611638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.621613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.622391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.622671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.622945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.633400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.635649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.637735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.643351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac67970-6416-4229-8795-2f9122a99adc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:15.662731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:15.663759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:15.664715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:15.669028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 275/537] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:15.672830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.723095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.723634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.723909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.724175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.724881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:15.725183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.730807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.750418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 263/538] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:15.756377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.799300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5407c0e-3927-4804-849d-068136c59c3f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.847795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.848562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.848944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.849319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.850121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:15.850554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.852092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:15.870665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 276/539] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:15.873306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:15.946605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:15.954581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a2138c1-dd93-4087-a114-fd6a456460fd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:15.991680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:15.992286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:15.992572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:15.992846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:15.993577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=cefcf250ee17414389d3014e8755416e) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:15.993888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:15.995356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2d218f-09d6-4be6-9691-9019f4951ff0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:16.001769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 264/540] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/role_assignments?user.id=cefcf250ee17414389d3014e8755416e&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:16.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87d0eb1-d121-4921-90dd-65778cb3099e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:16.091530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:16.092148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.092430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.092700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.093441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=5c3ad8bf2cc8457eb6685a514e545dbd) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:16.093753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:16.095195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbb587f-fb26-41fb-846f-54a449b1f5f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:16.106458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 277/541] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:15 2022] GET /identity/v3/role_assignments?user.id=5c3ad8bf2cc8457eb6685a514e545dbd&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:16.177521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be197a33-2856-461a-89ad-83ea697b8560 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.178047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be197a33-2856-461a-89ad-83ea697b8560 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.178047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be197a33-2856-461a-89ad-83ea697b8560 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.179827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 265/542] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:16.186707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148bbec-4f14-45fc-a670-dd34bbf59d8c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.187055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148bbec-4f14-45fc-a670-dd34bbf59d8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.187339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f148bbec-4f14-45fc-a670-dd34bbf59d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.211692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f148bbec-4f14-45fc-a670-dd34bbf59d8c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:16.213020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f148bbec-4f14-45fc-a670-dd34bbf59d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:16.290539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 278/543] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:16.779423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be197a33-2856-461a-89ad-83ea697b8560 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:16.792264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:16.883295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a754-7be2-453b-826e-ce98422a7e27 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.883607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a754-7be2-453b-826e-ce98422a7e27 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.883857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d1a754-7be2-453b-826e-ce98422a7e27 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.886417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 279/544] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:16.894836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d60ca-e036-48e3-8313-488f874682ce None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.895125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d60ca-e036-48e3-8313-488f874682ce None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.895401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057d60ca-e036-48e3-8313-488f874682ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.910043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be197a33-2856-461a-89ad-83ea697b8560 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:16.925236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-057d60ca-e036-48e3-8313-488f874682ce None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:16.926461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-057d60ca-e036-48e3-8313-488f874682ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:16.948420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:16.949060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:16.949361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:16.949664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:16.961191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:16.966817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:16.967730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:16.968521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 266/545] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:16 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:16.979195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.018644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.053779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 280/546] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:17.054342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.091886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a1e1-81b8-478e-8dbf-13097a314653 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.126811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.127415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.127688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.127956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.128619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:17.128915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:17.130298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:17.139448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 267/547] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:17.144599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d60ca-e036-48e3-8313-488f874682ce None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.297572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057d60ca-e036-48e3-8313-488f874682ce None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.333566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.334279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.334584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.334855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.346149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.346691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:17.348111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:17.364534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:17.365508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:17.366401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:17.366401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:17.366401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:17.366401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 281/548] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:17.372044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.418610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.485733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f9d8d9-d80e-4ce9-ad44-c348ae38296d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.527292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.536350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.537259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.537540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.537814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.541689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8af2a6-fa96-46f9-a839-aa6a25086a47 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.552535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f0459891-8f73-462a-90b0-c4af3059ef00 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:17.560781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:17.562173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:17.563217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 268/549] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:17.563900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb920-75f4-4f13-bd22-8eda077074f9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.570292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb920-75f4-4f13-bd22-8eda077074f9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.570292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66fb920-75f4-4f13-bd22-8eda077074f9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.570292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 269/550] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:17.571968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.572386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.572786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.579962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.580656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.581007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.581348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.582084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:17.582483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:17.583975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:17.596782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:17.597197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:17.604931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 282/551] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:17.610345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.703500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 270/552] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:17.704768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.732395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.761429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84f7599-7b1f-4259-ae9e-a41e6739f171 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.851392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240aa132-0eef-4db9-8983-d7ff7cdd01a5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:17.863259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.863877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.864152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.864417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.865096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:17.865456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:17.866870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:17.884715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 283/553] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:17.888264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:17.901203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:17.927647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:17.928288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:17.928568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:17.928840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:17.942680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:17.945599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:17.970545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:17.971641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:17.972585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 271/554] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/users/swift => generated 83 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:17.981651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.076264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f4d0e4-266e-45d8-8948-6416633d376e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.113576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.114575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.115082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.115570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.127364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.129151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.132648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d53e66-ebca-4e90-9149-2da32d0b1da8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:18.145470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:18.146795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:18.148209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:18.148209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:18.148209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:18.148209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 284/555] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:18.148682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.153400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.178391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.179378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.179840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.180430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.181508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:18.182007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.183808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.206090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 272/556] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:17 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:18.207335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.256987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.266006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73552d87-2bf6-403c-a5c8-937c6dc9c587 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.304854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.305599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.305953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.306325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.317946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:18.323910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:18.325982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:18.327324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 285/557] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:18.331211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.333423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbedc4b-27dd-4fc7-a1b4-45c3ea59ee05 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.376808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.377429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.377429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.377429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.378799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:18.378799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.379808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.392164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 273/558] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:18.392967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.406585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.452685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5577ec91-64df-43e4-9a43-b68ac87d3d28 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.490065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.490765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.491029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.491260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.500852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.504560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0786bb-9391-461a-a79e-a9e290b472eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.543206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.543817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.544099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.544366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.545029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:18.545324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.546730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e781f1-fea9-4461-a03c-21a6c73c0efb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.554849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 274/559] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:18.556011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d60f5-83b8-4cf2-ab8a-34e17b06a0ca None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.556289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d60f5-83b8-4cf2-ab8a-34e17b06a0ca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.556556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7d60f5-83b8-4cf2-ab8a-34e17b06a0ca None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.557730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 275/560] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:18.558434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d60f5-83b8-4cf2-ab8a-34e17b06a0ca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.564899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.566382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2197cc9e-169a-4150-86ce-e3f848da6176 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.627039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.661068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7d60f5-83b8-4cf2-ab8a-34e17b06a0ca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.665646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 286/561] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:18.666901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee088-36cb-4f36-83d9-556f54e250ac None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.667182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee088-36cb-4f36-83d9-556f54e250ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.667448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150ee088-36cb-4f36-83d9-556f54e250ac None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.668687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 287/562] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:18 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:18.669374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ee088-36cb-4f36-83d9-556f54e250ac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.698071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.698727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.698999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.700005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.707069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.708654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aabf2360-0f0b-4100-be75-48ce5f90e435 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:18.720069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:18.721046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:18.722464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 276/563] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:18.723284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.723284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.723284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.741736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:18.742949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:18.759296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.774445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150ee088-36cb-4f36-83d9-556f54e250ac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.807159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.807799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.808065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.808333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.811297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 277/564] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:18.812309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.831839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.833298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-213e61ad-d17b-4a09-b49a-a509a0427cc9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:18.848975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:18.849956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:18.850899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:18.850899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:18.850899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:18.850899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 288/565] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:18.852058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.852405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.852753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.871580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:18.872883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:18.884715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:18.924207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9adf1292-80cd-47f8-aef9-2ae12753ff12 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:18.950768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 289/566] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:18.950768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:18.968869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:18.969867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:18.970148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:18.970467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:18.985151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:18.987128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:19.006698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.007853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:19.008843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:19.008843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:19.008843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:19.008843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 278/567] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.008843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.017306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.078161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280aaa6e-7c24-425f-a4ea-ee3bae493ea3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.115022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.115625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.115933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.116464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.116879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.117177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.118599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.132425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841112b6-9388-4b99-bc8a-3ac2c7a160e1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.141391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 290/568] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.141391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.145637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.173650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.174338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.174573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.174842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.175483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.175784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.177148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.190200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 279/569] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.190894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.261489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67d5905-8d6d-4639-8c66-fe6df809d192 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.266755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.298602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.299206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.299493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.299759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.300397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.300700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.302088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.306169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff55ffc-9d9e-43af-902e-4ed5941dd58d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.313838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 291/570] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.314670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.342634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.343487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.343912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.344176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.353919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:19.358723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.359867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:19.360900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:19.362392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 280/571] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.363367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.383012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.423191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cff713e-15ab-4093-b784-c9d0eb488657 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.457189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.457815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.458096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.458409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.463432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87a3094-15ba-41d5-8209-8b67082bfb60 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.469054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.470529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:19.485297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.486244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:19.487257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:19.487257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:19.487257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:19.487257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 292/572] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.487481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.492063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.503739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.504386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.504663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.504930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] PATH_INFO: `/v3/users/glance` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.512158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.514140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:19.529432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.535576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 281/573] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/users/glance => generated 84 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.536791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.596990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770e081f-4312-4d8d-9b12-3a0530b0540a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.613691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.632738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.633366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.633640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.633910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.645229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:19.649677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48751cd2-e236-460b-b6b5-6f4d70776397 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:19.650719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.651814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:19.653363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 293/574] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.654082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.688463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.689012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.689289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.689558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.696526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.698016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:19.709513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:19.710644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:19.711552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 282/575] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:19.712108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.756217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4152712-c64a-456b-8222-56ed2cd99da7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.758084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.790091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.790712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.790984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.791250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.791900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.792199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.793674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.801344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 294/576] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.802121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.817941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d5798c-9410-42de-9f00-5b96437ca010 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.842272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:19.852756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.853355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.853630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.853895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.854556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.854852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.856243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.867657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 283/577] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.868454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.912060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77e56-e549-4638-8e53-b9a8a296c9f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.951208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:19.951741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:19.952019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:19.952291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:19.953025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:19.953331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:19.954787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:19.972605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 295/578] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:19.973429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:19.976344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ca3c01-3e29-4e9d-8d3d-3d9056bae61f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:19.978649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.012314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.012966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.013245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.013517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.014180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.014513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.015892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.024024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 284/579] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.024807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.073720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.080878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056e666b-8beb-4f46-9e1f-319265a21037 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.116024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.116622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.116899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.117169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.117859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.118162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.119590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.127448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a4ff10-c0d6-4e8b-aa8f-71cfd7debcda None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.132584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 296/580] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.133450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.162547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.163173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.163451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.163718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.175172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.176620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:20.192504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:20.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:20.194506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:20.194506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.194506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:20.194506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 285/581] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:20.194506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.218970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.243181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6f034dc-804b-408d-9f2d-3357580fd118 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.276968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.278572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.278572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.278828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.285362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.286821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:20.298371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:20.299392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 297/582] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:20.300273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.310981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637a78c7-3c1c-499a-8389-77060c4b08f6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.318461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.347720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.348432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.348762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.349085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.361815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.363649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:20.377508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:20.378584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:20.379483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:20.379483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.379483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:20.379483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 286/583] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:20.379483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.405576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7da32b8-98fd-49c8-ac73-e068579614ba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.439831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.440394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.440670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.440937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.450071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.451518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.452949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:20.467356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:20.468510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:20.469675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:20.469675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:20.469675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:20.473121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 298/584] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:20.473919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.482276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42fbf951-fcc6-4531-9197-c25c84b38fa7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.519167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.519776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.520056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.520328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.521061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=fb30d87ff47b49e39091ef401f10ff59) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.521368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.523102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.528583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 287/585] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/role_assignments?user.id=fb30d87ff47b49e39091ef401f10ff59&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.529457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.548169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.595764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25152e79-292d-4f09-a687-70fdabbba26d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.642753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.643404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.643690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.643966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.644662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.644966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.646392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.659621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 299/586] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.660496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.674658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbf3eb3-520f-4d12-af60-93b976d8e130 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.709792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.710597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.710972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.711341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.712130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.712553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.714127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.716054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.729565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 288/587] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.731109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.776249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7da487-a848-4c44-9e79-2771b5a8760d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.805259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:20.822459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.823152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.823516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.823859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.825389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.825836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.827218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.845574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 300/588] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.845574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.845838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3247109-5b5c-4a7a-a4be-d027b139a387 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.882007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:20.882641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:20.882920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:20.883182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:20.883817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:20.884111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:20.885460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:20.905418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 289/589] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:20.905418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:20.959919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996f8df8-534e-4f76-87e4-625ac3d8296b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:20.972382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.002364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.002947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.003227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.003505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.011344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.013545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.023216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71215ef9-15ce-421f-9cc3-e3bbeff858be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:21.026669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:21.027752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 301/590] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:20 2022] GET /identity/v3/users/neutron => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:21.028691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.064756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.065753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.066169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.066641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.083533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=148e575ef9b642c8b81d7eca8dd70bfa, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.084156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.085191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.098604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 290/591] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:20 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:21.099454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.156080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba14c358-baf0-4851-9656-540eb2d15c9e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.196664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.197574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.197993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.198563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.207269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.208996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:21.222833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:21.223968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:21.225037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:21.227285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 302/592] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:21.228357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.259645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55217496-a9e6-447e-b974-f2d4ec543fb4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.264302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.310476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.311303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.311657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.313037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.326099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.328520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:21.349684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:21.350756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:21.351746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:21.352564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 291/593] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:21.354054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.377420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfee123-9803-48d4-b9fc-e3375c48edaa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.388543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.416111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.416794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.417210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.417575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.419113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:21.419722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.421623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.441317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 303/594] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:21.445760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.483762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97871ea-77a7-48a7-92e8-c0d5667c2ba4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.522186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.523056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.523512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.523929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.524766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:21.525212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.526912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.540961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 292/595] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:21.542101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.553833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a9f65-b07d-4881-a572-2ff41477967a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.570939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.593865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.594658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.595050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.595451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.596427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:21.596824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.598525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.620189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 304/596] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:21.622011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.658879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.660532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbc54a5-5a88-4a90-9474-b2727c1cdac1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.704501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.705139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.705416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.705747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.717529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.719132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:21.735017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:21.736181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:21.737315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:21.737315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:21.737315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:21.740979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 293/597] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:21.741823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.745052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53e567-d997-4847-8ce3-932e955d3f3e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.782103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.782898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.783277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.783670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/288a7a6690d3480db235d2f594bc0acd/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.799612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=288a7a6690d3480db235d2f594bc0acd, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:21.801182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:21.830295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 305/598] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:21 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/288a7a6690d3480db235d2f594bc0acd/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:21.830295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:21.849581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:21.905299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd093ad7-69c3-46aa-8e15-3f2b1565dab7 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:21.969781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:21.970451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:21.970731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:21.971004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:21.976462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.003275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.004813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.006643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e43962d-2e9a-45a6-86e3-75413b6ab50e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:22.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:22.032622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:22.033546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:22.033546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:22.033546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:22.042758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 294/599] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:22.044707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.052231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.053137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.053621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.054135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.055043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:22.056017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.057672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.074551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 306/600] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:22.080528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.183969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba42fb5-6433-4a0d-9ea7-1d621d047287 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.212960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7192dd-b5b9-4117-835a-4f1bdb48f7f4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.247423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.254433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.255085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.255364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.255635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.264360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.266002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:22.279074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:22.280012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:22.280905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 307/601] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:22.282374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.283008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.283591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.283864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.284244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.284794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:22.285088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.286601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.287673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.308233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 295/602] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:22.319410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.399884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf0f8ca-9547-4cf7-b284-ab21b6f1c198 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.448366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.449514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.449983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.450456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.451346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:22.451854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.453464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.467106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960d413c-9689-4288-b8c3-3f23af4f7c61 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.479869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 308/603] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:22.485696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.529420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.530096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.530351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.530619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.540745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.542131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c9af19e0-d56f-46af-a734-48ef799da8d0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:22.567013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:22.567962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:22.568861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:22.570421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 296/604] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:22.571557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcba7-cb05-4221-8584-12d687de5119 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.571832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcba7-cb05-4221-8584-12d687de5119 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.572096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fcba7-cb05-4221-8584-12d687de5119 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.574913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 297/605] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:22.578658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fcba7-cb05-4221-8584-12d687de5119 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.586910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.638564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.640929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e93cd3-e8fd-4a36-8040-34efcdec118a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.678322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.678980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.679255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.679517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/5c3ad8bf2cc8457eb6685a514e545dbd/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.696875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fcba7-cb05-4221-8584-12d687de5119 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.697753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=5c3ad8bf2cc8457eb6685a514e545dbd, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.699994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e88fc8cd-ab5b-43ea-bb13-95052844ef66 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.738548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 309/606] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:22 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/5c3ad8bf2cc8457eb6685a514e545dbd/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:22.741528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.741813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.742072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.745303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.746502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.747034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.747568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.748630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:22.749186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:22.751101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:22.765707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 298/607] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:22.771965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:22.774906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:22.774906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:22.857989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 310/608] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:22.940159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c6eeb4-54ee-4308-abbf-31654df95279 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:22.954655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:22.981319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:22.982082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:22.982457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:22.982734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/cefcf250ee17414389d3014e8755416e/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:22.999859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=cefcf250ee17414389d3014e8755416e, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:23.001311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:23.018176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 299/609] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:22 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/cefcf250ee17414389d3014e8755416e/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:23.176966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:23.218852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:23.290967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff24f1ba-f6b9-4931-8650-7cfbf8233d63 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:23.329591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:23.330220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.330529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.330801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.345432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5e198de8-a6cb-47d9-b09d-9013a0d21d04 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:23.350438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:23.351369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:23.352595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 311/610] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:23.358578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:23.399877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbbcf9-dd02-48fa-9920-59f99155497e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.400228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbbcf9-dd02-48fa-9920-59f99155497e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.400413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfbbcf9-dd02-48fa-9920-59f99155497e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.401993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 312/611] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:23.408188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.408472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.408759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.430447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:23.431559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:23.494083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cb59cd9-56a5-44e9-83fc-0b825a62d0cd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:23.531352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:23.532007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.532238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.532483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.533150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:23.533447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:23.534843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:23.544433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 300/612] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:23.549825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:23.572927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:23.615910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 313/613] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:06:23.659197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58088133-8080-4249-9301-24e46ae8d26c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:23.696275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:23.696906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.698291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.698291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.712678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:23.714607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:23.730920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:23.732198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:23.733122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:23.733122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:23.733122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:23.734978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 301/614] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:23.740811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:23.778789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:23.850576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5721a21c-7075-46fd-8c6e-b9ac28dfa701 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:23.887466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:23.888423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:23.888882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:23.889332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:23.890274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:23.890758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:23.892351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:23.910408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 314/615] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:23.912141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:23.987189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.018928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e64bfd-19fe-46a6-8bd4-1afad3b498fa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.053942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.054893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.055352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.055800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.065913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.069546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:24.090842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:24.091857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:24.092807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:24.094858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 302/616] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:23 2022] GET /identity/v3/users/swift => generated 83 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:24.100739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.122931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.211900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.227943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b38a472b-3be4-49c8-9a7f-7a874999123f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.303340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.303870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.304149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.304415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.305157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:24.305458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.308375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ea43c0-014d-4f3c-b90a-84fb737e5430 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.310915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0f10b3-ca7d-4f79-88a5-1535f8bd07dc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.331981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 315/617] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:24.333484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e608-5fee-4483-a15b-401122cfda1d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.333872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e608-5fee-4483-a15b-401122cfda1d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.334276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c425e608-5fee-4483-a15b-401122cfda1d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.340336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 316/618] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:24.341555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425e608-5fee-4483-a15b-401122cfda1d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.349138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.356980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.357638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.358300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.358584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.372481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f4b3bf01-ab6d-4052-9873-81ca581b7ddd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:24.379575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:24.380537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:24.381454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:24.382523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 303/619] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:24.383474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.383753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.384022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.404714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:24.405981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:24.468914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c425e608-5fee-4483-a15b-401122cfda1d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.483576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 304/620] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:24.484923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.505558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.506189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.506509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.506778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.518014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.519399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.521169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:24.535493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:24.536519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:24.537718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:24.537718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:24.537718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:24.537718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 317/621] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:24.543709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.599000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eee16b-8253-4e22-9523-ddea3fe9de32 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.626697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.638492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.639527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.640023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.640485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.641434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:24.641945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.643632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.657908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 305/622] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:24.659538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.663627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0283142-e482-4c4f-a286-fc33167da12e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.716031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.717087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.717573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.718037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.718897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:24.719492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.721032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.747661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 318/623] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:24.754632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.802035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c2698e-0eb4-451d-9d57-00184442ccb3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.851104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.853121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.853395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.853658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.870046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.873234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:24.885455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:24.895796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:24.897027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:24.897913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:24.897913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:24.897913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:24.900364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 306/624] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:24.901298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:24.946961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e665c5-d07e-4e67-b485-dc62700a2e34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:24.951497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:24.984190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:24.984950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:24.985232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:24.985496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:24.993003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:24.994612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:25.007641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:25.008601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:25.009508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 319/625] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:25.010002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.039065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5076b5f2-810d-48c5-92d9-ba1feb66d0fc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.086790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.088631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.089136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.089641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.104028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:25.112673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:25.113665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:25.114562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:25.120469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 307/626] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:25.120469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.129110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a79ec89-e994-4594-b990-7dbd78701103 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.147154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.175176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.176405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.177087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.177509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.178327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:25.178904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.181060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.197354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 320/627] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:25.198472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.246852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.253692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb86e0bb-2dd2-4119-a809-c2ba56680574 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.292103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.293253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.293737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.294208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.295291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:25.295797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.297463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.311434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 308/628] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:25.312751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.331492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe739e3-c3cf-4c81-9093-12a24cbc9edd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.377928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.379204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.380091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.380698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.381609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:25.381931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.383348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.393014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 321/629] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:25.393801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.432163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c48bb2-4ecf-4d0d-a995-df46a9dd3d23 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.435376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.471295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.471923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.472204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.472478 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.480002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.481646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-30e7118a-fb6f-44b8-9b95-8eac61b39095 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:25.494865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:25.496397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:25.497362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 309/630] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:25.498551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577082-9495-46e0-a056-4eb96c874f2f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.498551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577082-9495-46e0-a056-4eb96c874f2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.498551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25577082-9495-46e0-a056-4eb96c874f2f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.499580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 310/631] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:25.500078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25577082-9495-46e0-a056-4eb96c874f2f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.515833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efcf1e0-d0b4-4c75-8621-235352fe18bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.519482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.550743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.551398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.551680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.551954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/fb30d87ff47b49e39091ef401f10ff59/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.566953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=fb30d87ff47b49e39091ef401f10ff59, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.568388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.581972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 322/632] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:25 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/fb30d87ff47b49e39091ef401f10ff59/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:25.582820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.618463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25577082-9495-46e0-a056-4eb96c874f2f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.660779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.661450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.661795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.662136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.674312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.675800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.684147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2b702f87-2a28-4fc6-af65-8b0ba1e4667c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:25.692590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:25.693648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:25.694610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:25.694610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:25.694610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:25.694610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 311/633] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:25.695153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.695312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.695583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.708269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-675473ec-1eba-4cc2-9996-d76bde1862bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.718739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:25.719967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:25.751849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.752471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.752750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.753025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.753756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:25.754060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.755499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d08003a-20d5-4f85-a990-47ce4961924d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.774290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 323/634] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:25.779372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b90e0-49fa-4ce9-84c7-90754c7ebf20 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.779372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b90e0-49fa-4ce9-84c7-90754c7ebf20 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.779372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86b90e0-49fa-4ce9-84c7-90754c7ebf20 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.779372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 324/635] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:25.779372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b90e0-49fa-4ce9-84c7-90754c7ebf20 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.796680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.806414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 312/636] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:25.806414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:25.892962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86b90e0-49fa-4ce9-84c7-90754c7ebf20 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.928914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.929442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.929721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.929989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.930670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:25.930973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:25.932421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1950e3-d695-47f2-a594-cdd170138612 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:25.938144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdccb14-4320-4c45-98ff-0b07a192eaef None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:25.948893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 325/637] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:25.948893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.948893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.948893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:25.954002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:25.966424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:25.967621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:25.986485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:25.987515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:25.988038 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:25.988554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.049234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 326/638] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:26.050199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.062063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.068365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.070208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.181506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0e12bf-bf4d-4dbb-bcd2-45d40db494c7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.189322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 313/639] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:26.190293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.225952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.226829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.227207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.227567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] PATH_INFO: `/v3/users/glance` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.229909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.237835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.239622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:26.252241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:26.253272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Aug 30 04:06:26.254215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:26.260996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 327/640] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/users/glance => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:26.262735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.309024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.315238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db77ac4-1a82-42ea-a2ed-42baae7811ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.358240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.359262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.359751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.360357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.373809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:26.380008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:26.380984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:26.381843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 314/641] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:26.382621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.391963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8add970-e95b-4d0e-ab2d-9a58047050bb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.431690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.432351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.432629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.432900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.446795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.448374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:26.464048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:26.465428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:26.466921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:26.466921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:26.466921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:26.472168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 328/642] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:26.472168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.494791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31ec2ac-e6d3-48c6-b11f-4d5d44b19573 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.522976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.533450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.534136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.534502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.534838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.535631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:26.536118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.537430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.556927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 315/643] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:26.557576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.577276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.584757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ad1e0d-0d51-4722-aa33-d8022d4cc4b6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.621624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.622318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.622598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.622866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.633479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:26.639492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:26.642330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 329/644] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:26.643275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.674108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0b85f4-7d9e-48c6-a472-a9d2db3d2e91 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.711915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.712438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.712717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.712970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.713627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:26.713922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.715901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.728590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 316/645] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:26.728590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.754607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea2ed0b-b2e3-43c1-9a74-1e8fa80537b1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.761785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.790994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.791463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.791835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.792204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.793551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:26.793912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.795367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.812857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 330/646] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:26.813911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:26.817306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:26.841916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08b8d16-15d9-4720-bf47-7353ff3a598d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.885038 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.885651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.885933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.886208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.927998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17655ec7-f864-4eb7-af79-590775f99b73 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:26.961547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.963375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.967915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:26.968530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:26.968813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:26.969087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:26.969756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:26.970060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:26.971494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:26.980199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 331/647] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:26.981438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.013187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.065253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.091239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 317/648] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:27.092093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.092975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94983fb3-f0c5-46a7-af6a-0631e8e51eb6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.134192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.134881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.135678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.135945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.148654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.150113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:27.165304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:27.166352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:27.167241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:27.167241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.167241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:27.167241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 332/649] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:27.167241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.209028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0b376d-a544-4e29-99e6-d935a2563fcb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.244686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.245659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.246105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.246541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.253391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.255003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.258569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:27.267966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:27.268899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 318/650] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:27.269805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.286135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10a70c0-8aa0-4a72-9a4d-e796c1608467 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.312809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.323486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.324707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.324988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.325260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.336010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.337435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:27.352339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:27.353502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:27.354737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:27.354737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.354737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:27.358339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 333/651] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:27.359202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.381808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76e839-a6f0-40a8-b519-cb9b7e3f56d3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.421152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.422270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.422569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.422841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.435214 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.436669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:27.452735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:27.453985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:27.455461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:27.455461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:27.455461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:27.455461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 319/652] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:27.456998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.471091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6766d262-bb15-46e0-aec7-151a415bfa13 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.502470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.508978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.509685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.510038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.510415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.511064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:27.511989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.513491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.531944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 334/653] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:27.531944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.557997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.568803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6ad248-ac99-4010-a49c-d20ac0df32f8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.602839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.603444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.603721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.603993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.604686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:27.604994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.607134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.619426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 320/654] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:27.620651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.632763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c7bf127-2438-4f67-b710-2880d1f16764 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.666902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.667506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.667944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.668366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.668852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:27.669152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.670547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.682808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 335/655] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:27.686844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.732621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d300de9f-12ca-42aa-8d72-5d9791651969 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.744635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.768081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.768609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.768893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.769172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.769809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:27.770110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.771795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcac974d-1a42-44c6-b6c3-a704272f7bbf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.783796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 321/656] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:27.784980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea1f23-7ccc-47c6-b2cf-f6a9ab6a1cb8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.785613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea1f23-7ccc-47c6-b2cf-f6a9ab6a1cb8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.785893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04ea1f23-7ccc-47c6-b2cf-f6a9ab6a1cb8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.790293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 322/657] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:27.790293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea1f23-7ccc-47c6-b2cf-f6a9ab6a1cb8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.796210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e47598-3a70-4ab7-8c4f-983b9592a3aa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.801640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.830586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.831210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.831487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.831756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.839127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.841107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:27.851986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:27.852983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 336/658] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/users/cinder => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:27.853935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.899025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04ea1f23-7ccc-47c6-b2cf-f6a9ab6a1cb8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.933401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.933992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.934278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.934559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:27.935273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:27.935576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:27.936993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:27.942727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 323/659] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:27.943268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:27.957003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f3c3f4-7017-4a39-9ace-a1fd7d7a3dd9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:27.972717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:27.998503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:27.999160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:27.999435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:27.999703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.008125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.009780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b89256cb-2077-485e-853c-a48e286833c2 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:28.023096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:28.024119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:28.025117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 337/660] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/users/neutron => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:28.025720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.026180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.026180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.046111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:28.047361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:28.051890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.084046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f46ae0-8109-4311-b01c-d1fa75fc0fa4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.127065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 338/661] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:28.128351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.130616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.131256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.131537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.131806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.140051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.141547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:28.154070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:28.155204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:28.156367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:28.160901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 324/662] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:28.160901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.257724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d1e4ef-0fc6-47a6-80a1-b452b9b4134b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.259618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.282625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1831b6ab-5426-494c-958f-fc587d5acbc4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.306618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.307707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.308721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.309056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.313459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:28.313871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.316512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.320971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.321582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.321953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.322318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.323067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:28.323428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.326348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.328156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.345068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 339/663] 10.222.0.46 () {56 vars in 1137 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:28.346593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.348468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 325/664] 10.222.0.46 () {56 vars in 1139 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:28.349563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.460723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2addb215-40b9-40b3-bb85-44f0e88ea859 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.461638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c85dfa1-f0b5-4c52-84f2-82d7058e77f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.498627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.499233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.499510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.499686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.500124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.500308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.500583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.500854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.501554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:28.501861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.503544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.516990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 340/665] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:28.517830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.550808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.569614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.582330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.584026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.637025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d167fd1-3d05-445f-97a9-06992fe691b5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.674198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.674967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.675318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.675596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.691660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 326/666] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:28.691660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.752799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:28.754231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:28.776720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.782666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:28.793459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e76b0c2-2194-430d-933f-010b2be885ea None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.835173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:28.835804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:28.836083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:28.836356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:28.846654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:28.851502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:28.852420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:28.853267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 327/667] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:28.853997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.879692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 341/668] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:28.880637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:28.978504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6ee9717-972e-40c7-bd0a-3857c2e0237a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:28.996417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85907283-cc2a-4178-82ec-e596f80c30a2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.015634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.016233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.016511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.016781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.017500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=288a7a6690d3480db235d2f594bc0acd) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:29.017804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.019266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.019396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.026134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 328/669] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/role_assignments?user.id=288a7a6690d3480db235d2f594bc0acd&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:29.027027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.048237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.048943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.050722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.051238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.051703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.069603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.072697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:29.095263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:29.096364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:29.097365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:29.097365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.097365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:29.104312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 342/670] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:29.104312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.155169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70dc5936-c1e8-433b-8099-0e9877a2bb71 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.203216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.203863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.204143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.204416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.205192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:29.205501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.207036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.218560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 329/671] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:29.219242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.268359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2d89df-5fed-4b49-b327-ffb7d8e19fe0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.306358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.307362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.307644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.307910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.316306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.321019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.322563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:29.341583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:29.342728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:29.343784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:29.343784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.343784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:29.343784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 343/672] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:29.343784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.349564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13197bc-a951-41cc-87c7-f857b5c6511c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.374732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.389890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.390522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.390871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.391215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.391937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:29.392399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.394056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.411470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 330/673] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:29.412752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.462598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51fc9f-1936-4a32-a857-78d29d903bac None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.499581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.500220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.500753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.500753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.513494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.515055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.523168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86867577-97f0-4cd9-bfe6-e4562d817c36 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:29.534287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:29.535315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:29.536216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:29.536216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:29.536216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:29.539126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 344/674] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:29.539126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.562900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.563561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.563842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.564113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.564764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:29.565068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.566537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.573541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.581819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 331/675] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:29.583930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.635833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:29.670991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a00084-a21f-417d-b0c7-e0c0622af35c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.720604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.721314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.721598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.721871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.732081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.733606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.748634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583dd2a8-e0bf-4446-8670-797b1c44326b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:29.753552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:29.754689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:29.755604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:29.765330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 345/676] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:29.766226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.875196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:29.875895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:29.876104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:29.876375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:29.877032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:29.877333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:29.878742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:29.916678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 332/677] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:29.916678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:29.952783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308b6ca8-faeb-4bcd-a6c6-74005fb99488 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.022978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.024399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.024639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.024913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.030761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.030906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.043034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.044740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:30.057751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:30.059435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:30.060540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:30.064708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 346/678] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:30.065575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.090895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a38fc2a-acd5-4217-b93c-98dd7efa1868 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.140842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.141382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.141664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.141977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.142882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:30.142992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.144373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be51a2f4-fe9c-40e8-a0d8-b6c33e56e41d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.157928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 333/679] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:30.158986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162dd3-504d-4ce9-a1ed-62f9edf61cd6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.159277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162dd3-504d-4ce9-a1ed-62f9edf61cd6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.159548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52162dd3-504d-4ce9-a1ed-62f9edf61cd6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.164772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 334/680] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:30.164772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162dd3-504d-4ce9-a1ed-62f9edf61cd6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.197026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244d9805-377e-4ae6-a408-db2ff91edb5a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.232792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.233428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.233703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.233974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.241974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.243438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:30.255352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:30.256508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 347/681] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/users/swift => generated 83 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:30.257495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.283183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52162dd3-504d-4ce9-a1ed-62f9edf61cd6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.302134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.321654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.322932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.323518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.324730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.325019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.326297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:30.326675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.329073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbffdf3e-52ae-4591-813c-c3b319024403 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.343760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 335/682] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:30.349733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.350025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.350330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.370119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:30.372362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:30.373052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec38fea3-1898-4ad4-868a-60b952627c7d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.412201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.412822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.413104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.413376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.414104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=cefcf250ee17414389d3014e8755416e) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:30.414444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.415869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.430386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 348/683] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/role_assignments?user.id=cefcf250ee17414389d3014e8755416e&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:30.430386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.457750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 336/684] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:30.458710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.570104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e064a9d-d72c-42bb-8d87-595090488707 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.582105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e72d7a57-9c12-4f3d-867b-80caa7a40da7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.594336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.600409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.619871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.620496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.620814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.621182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.622075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:30.622417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.624474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.648801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 349/685] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:30.657907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.662801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.663521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.663949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.664319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.665153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=5c3ad8bf2cc8457eb6685a514e545dbd) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:30.665561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.667257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:30.676569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 337/686] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/role_assignments?user.id=5c3ad8bf2cc8457eb6685a514e545dbd&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:30.779236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4656cb-813a-4d86-b8d8-033666235e53 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:30.836430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:30.837454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:30.837943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:30.838476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:30.868957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:30.894172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:30.922415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:30.923923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.028987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d08553-247c-46da-9f31-1e2b7ea48d18 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.052800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 350/687] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:31.059359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.090244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.091114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.091365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.091638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.123340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:31.129168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:31.130124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:31.131035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:31.137169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 338/688] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 266 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:31.138260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.151147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.176918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c5076d-754d-4949-98e5-571794117c16 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.187403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.234017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.237306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.238482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.239139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.254555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.256365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-592da6d5-5d49-4aa8-b89f-b849813d7098 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:31.274334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:31.275366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:31.276282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:31.276282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.276282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:31.279484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 351/689] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:31.285519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd73e-94c3-403b-9dc6-c939aa122a5d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.286061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd73e-94c3-403b-9dc6-c939aa122a5d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.286556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12cd73e-94c3-403b-9dc6-c939aa122a5d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.286738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7df418-d706-40bc-a90c-8ea7ee9fdd34 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.288655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 352/690] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:31.289601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd73e-94c3-403b-9dc6-c939aa122a5d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.326877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.327487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.327766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.328034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.328694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:31.329044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.330484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f604be3d-e7f9-412b-b218-a7a81c873f0a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.340106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 339/691] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:31.341737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.342026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.342370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.361927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:31.363134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:31.409516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12cd73e-94c3-403b-9dc6-c939aa122a5d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.440665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.443423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 340/692] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:31.444735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.447863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.450162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.450623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.450985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.451871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:31.452288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.453739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.471035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 353/693] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:31.479215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.480498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.613420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1feb42e8-72ed-446f-94d9-1d4f96649c9b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.658886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5329c3be-8594-4dca-9707-b454e2c0c53a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.668744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.669675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.670118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.670551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.687217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.688904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:31.716605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:31.717779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:31.718906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:31.718906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.718906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:31.722652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 341/694] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:31.726174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.727007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.727985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.731253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.731772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.746415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.748089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:31.765025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:31.766186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:31.767325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:31.769272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 354/695] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 290 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:31.770183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.847867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.866695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:31.875492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fda046e-2c20-464c-8128-5dce540adbc3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.905681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73bdb7dd-4743-4afe-a1b8-bd52cafd42af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:31.913739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.914662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.915154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.915625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.916583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:31.917084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.919509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.933676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 342/696] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:31.934978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:31.953544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:31.954177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:31.954495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:31.954763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:31.973871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:31.975290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:31.994690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:31.996296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:31.997211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:31.997211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:31.997211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:32.001741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 355/697] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:32.002871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.050153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7715a9c-6920-4e46-aa52-792661b26b4d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.086449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.087059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.087341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.087614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.088313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:32.088617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.091079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.103495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 343/698] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:32.104413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.109070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.136852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d703da1-03e3-4e65-8fc0-6bd5c0fa1134 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.138530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.175758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.176434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.176716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.176987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.185197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.186753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:32.199214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:32.200466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:32.201444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 356/699] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:32.202070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.218967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2476b2-4a24-4bd9-92f9-0e8759a1e994 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.254279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.254825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.255112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.255393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.256051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:32.256361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.257837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.274580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 344/700] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:32.275463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.314344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167f31fa-391e-47df-a152-67bab51deed5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.340307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.351847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee0376-3af5-480a-904a-283248424065 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.352457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee0376-3af5-480a-904a-283248424065 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.352739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee0376-3af5-480a-904a-283248424065 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.353068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdee0376-3af5-480a-904a-283248424065 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.353798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee0376-3af5-480a-904a-283248424065 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=fb30d87ff47b49e39091ef401f10ff59) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:32.354154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee0376-3af5-480a-904a-283248424065 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.356721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdee0376-3af5-480a-904a-283248424065 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.367566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 357/701] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/role_assignments?user.id=fb30d87ff47b49e39091ef401f10ff59&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:32.368397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee0376-3af5-480a-904a-283248424065 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.388387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e0b47-66fe-48fc-9038-89f5a65bfd30 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.401736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.430571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.431197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.431480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.431753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.432660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:32.432900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.435070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.456702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 345/702] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:32.462590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.535475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdee0376-3af5-480a-904a-283248424065 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.595844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.596471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.596750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.597015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.600785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.604705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2e3429-c9a0-4292-9b5a-277c64db40f0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.608285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99096aae-43cf-47d8-a6a5-5f4f6ac8cebe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:32.632168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.643251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 358/703] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:06:32 2022] POST /identity/v3/users => generated 254 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:32.661819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:32.662510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.662795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.663070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.746602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:32.748189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:32.882120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 346/704] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Aug 30 04:06:32.886373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f834-7f12-43b7-892d-1bef747120dd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.886856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f834-7f12-43b7-892d-1bef747120dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.887094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd2f834-7f12-43b7-892d-1bef747120dd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.888708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 359/705] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:32.889716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:32.896051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:32.896559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:32.896824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:32.907458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:32.925643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:32.926897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:33.009023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 360/706] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:06:33.023562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-481a167f-6ff1-41db-9578-2c0a76b3955b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.072557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.073174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.073404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.073750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.085348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.086904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c136acf5-0a1c-446e-8040-994ba8a6896d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:33.104574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:33.106521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:33.107440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:33.107440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:33.107440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:33.107440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 347/707] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:33.113659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.129560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.143894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6587f6-dedb-4617-b0eb-390fc614bafa None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.144228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6587f6-dedb-4617-b0eb-390fc614bafa None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.144433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6587f6-dedb-4617-b0eb-390fc614bafa None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.145894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 348/708] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:06:33.152288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.152590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.153030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.173674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:33.174910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:33.230861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7353a2e-f4af-49dc-bcfc-06c078e0a47e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.250054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.262356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 349/709] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Aug 30 04:06:33.267299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.267922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.268194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.268457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.269203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:33.269508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.270934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.283492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 361/710] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:33.289051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.361860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.466624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc85a1-cefb-4177-904a-a57eb27b8ba5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.515844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.543539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.544183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.544463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.544739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.548684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76248745-31bb-4364-a8f9-d75a47f54f2d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.563382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.564876 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.584202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.584829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.585150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.585423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.586005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.587372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:33.596234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:33.597149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:33.598052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 350/711] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:33.602288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:33.603320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:33.604162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 362/712] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:33.606497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.613891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.674882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.731495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e562204b-473b-47a2-9bc4-37cf84be333b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.749664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6848262-01f4-4210-a3b1-320932af777a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.768550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.769576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.770297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.770804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] PATH_INFO: `/v3/services/network` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.771882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.773495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:33.788829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:33.789979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:33.790865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:33.794903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 351/713] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/services/network => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:33.796068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.804965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.805588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.805868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.806140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.807649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:33.807970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.810620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.814215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.832510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 363/714] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:33.832510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:33.942765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b4a42-2851-4457-9feb-cb941ab939ce None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.966082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:33.967279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e00dc2-fd27-4110-8202-b9ac158ef3d4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:33.983556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:33.984466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:33.984978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:33.985455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:33.986347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:33.986855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:33.988432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:33.998638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 352/715] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:34.000091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:34.012583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:34.013212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.013496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.013925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.014598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] RBAC: Adding query filter params (name=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:34.015310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:34.016999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:34.026337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 364/716] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/services?name=network => generated 124 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:34.027675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:34.104762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:34.114439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369e2952-e512-4727-b18f-6e070421d802 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:34.154233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee0806-6516-47e3-a44a-c36ff7c94c77 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:34.155807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:34.156598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.156953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.157295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.158304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:34.158628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:34.160249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:34.173140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 353/717] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:33 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:34.174447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:34.207201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:34.207751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.208206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.208484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.209130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:34.209533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:34.210936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c41002c-d8de-415f-acbe-4f038f5b79b2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:34.226375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 365/718] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:34 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:34.251873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:34.328335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54deec6-85c4-4b5c-85ff-f4d768998db8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:34.369597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:34.370339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.370704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.370951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.371614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:34.371919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:34.373342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c0f97-f6ca-4e6c-acd5-2a78cc623431 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:34.380497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 354/719] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:34 2022] GET /identity/v3/services?type=network => generated 124 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:34.867720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e433d9-c75b-4b10-aaa2-e10c5e2b9a80 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.868010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e433d9-c75b-4b10-aaa2-e10c5e2b9a80 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.868294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e433d9-c75b-4b10-aaa2-e10c5e2b9a80 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.869941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 366/720] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:34.875418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d756-4823-4abc-bd37-10908a34eb65 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:34.875705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d756-4823-4abc-bd37-10908a34eb65 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:34.875981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e70d756-4823-4abc-bd37-10908a34eb65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:34.911638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1e70d756-4823-4abc-bd37-10908a34eb65 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:34.912856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e70d756-4823-4abc-bd37-10908a34eb65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:35.001802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 355/721] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Aug 30 04:06:35.189410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45277c-d7ca-40da-9afa-6087d5006bac None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:35.189953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45277c-d7ca-40da-9afa-6087d5006bac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:35.189953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b45277c-d7ca-40da-9afa-6087d5006bac None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:35.191322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 367/722] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:35.196678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:35.197100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:35.197300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:35.219313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:35.220513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:35.273420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:35.320508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 356/723] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:35.347752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45277c-d7ca-40da-9afa-6087d5006bac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:35.503563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b45277c-d7ca-40da-9afa-6087d5006bac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:35.535745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:35.554966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:35.556044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:35.556543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:35.557017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:35.575135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:35.580753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:35.581772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:35.582723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:35.585306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 368/724] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:35.591461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:35.666777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:35.774190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d96a30b-a28d-48d1-8bfa-07504148b11b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:35.848381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2b9b51-1b07-4011-a219-ce7ed3d92f51 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:35.850913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:35.851554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:35.851823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:35.852087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:35.852790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:35.853088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:35.854581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:35.873960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 357/725] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:35.883208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:35.907038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:35.907830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:35.908209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:35.908582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:35.920799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:35.926753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:35.927716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:35.928565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 369/726] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:35.936071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:35.954840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.024589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cf896d1-d129-4ae3-8444-04f0eac0b980 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.027976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.066179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.066874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.067154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.067422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.089661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.092442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:36.125664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:36.126748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:36.127673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:36.127673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:36.127673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:36.133890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 358/727] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:36.140866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67e4f18-e513-40a4-8847-591d9c4f364f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.148180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.207889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.208594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.208764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.209021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.209683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:36.209980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.211370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.221316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 370/728] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:36.230993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.294803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.311507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2bed8c9-5c52-4632-afaa-cb991486578a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.349966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.350616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.350898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.351445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.352112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:36.352417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.353811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.368232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 359/729] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:36.374432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.460532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af518120-7940-421a-8dea-c56ed96be04b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.471506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.505261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.505905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.506196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.506509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.520278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.522164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:36.539806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:36.540893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:36.541816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:36.541816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:36.541816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:36.541816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 371/730] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:36.549159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.598283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc1009a-8384-4d88-87e6-e2349ed2fd98 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.647023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.648243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.648773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.649054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.659921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.661771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7813068a-e0e4-4e84-9264-234d0b7a2934 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:36.679228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:36.680256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:36.681224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:36.686239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 360/731] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/users/swift => generated 83 bytes in 306 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:36.688178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1926f7-ff8c-490c-bd4c-ec768a645549 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.688467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1926f7-ff8c-490c-bd4c-ec768a645549 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.688742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1926f7-ff8c-490c-bd4c-ec768a645549 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.694368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 361/732] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:36.698375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1926f7-ff8c-490c-bd4c-ec768a645549 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.711417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.733302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050223-7b20-47e5-b7df-4060e0e07c06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.783724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.784689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.785196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.785649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.786499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:36.787058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.788508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdb33f5-682b-4955-aad4-bdc250efe82e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.807320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 372/733] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:36.809971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4d89-7d7c-4cf3-9114-def7cb363082 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.810246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4d89-7d7c-4cf3-9114-def7cb363082 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.812658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-679e4d89-7d7c-4cf3-9114-def7cb363082 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.831374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-679e4d89-7d7c-4cf3-9114-def7cb363082 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:36.832580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-679e4d89-7d7c-4cf3-9114-def7cb363082 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:36.840363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1926f7-ff8c-490c-bd4c-ec768a645549 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:36.847905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:36.880215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:36.880809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.881085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.881354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.882081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:36.882414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:36.883836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38059271-4e44-484b-bce0-74b2a5764e58 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:36.901515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 362/734] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:36.902922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb39034-e108-4801-a260-633ff3f364b1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.903207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb39034-e108-4801-a260-633ff3f364b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.903476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb39034-e108-4801-a260-633ff3f364b1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.904689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 363/735] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:36.905378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb39034-e108-4801-a260-633ff3f364b1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.913747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 373/736] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:36.915148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234247-c1b7-4c2e-b765-07924153110a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:36.915518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234247-c1b7-4c2e-b765-07924153110a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:36.915698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba234247-c1b7-4c2e-b765-07924153110a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:36.917078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 374/737] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:36.918421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234247-c1b7-4c2e-b765-07924153110a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:36.973419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.017222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb39034-e108-4801-a260-633ff3f364b1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.038243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba234247-c1b7-4c2e-b765-07924153110a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.055187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.055816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.056102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.056397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.064840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.066223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.074443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.075047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.075315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.075575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8d8445f5-e2ee-4fbb-8678-66e0606a215c None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:37.078880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:37.079832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:37.080776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 364/738] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:37.081846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a638b-9778-42c0-a85d-c995fbfbb775 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.081846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a638b-9778-42c0-a85d-c995fbfbb775 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.081846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435a638b-9778-42c0-a85d-c995fbfbb775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.099451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.099625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-435a638b-9778-42c0-a85d-c995fbfbb775 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:37.100652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-435a638b-9778-42c0-a85d-c995fbfbb775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:37.155016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.156464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.174554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 365/739] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:37.175814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.176101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.176375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.195402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:37.196760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:37.254020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.273863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 366/740] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Aug 30 04:06:37.274723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.297656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 375/741] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:37.299303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.365493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.379017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa8c530-360e-4286-b21a-dd9c29cfbdff None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.416299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.417040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.418067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.418375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.419264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:37.419642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.421140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.431775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0812eed-4e93-41cd-bfac-47b49130630a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.445226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 367/742] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:37.446230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.477010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.478215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.478523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.478942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.490633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:37.492527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:37.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:37.496747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:37.497597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 376/743] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:37.499142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.559948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76594ec4-7028-4e5d-a214-373b93c13fab None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.596590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.597291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.597571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.597861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.603947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3603bcf0-475e-4e3d-ac5d-dc5c77d8ad52 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.607961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.609401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.613335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:37.623999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:37.625111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:37.626027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:37.626027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:37.626027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:37.626027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 368/744] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:37.627219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.640362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.640974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.641255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.641531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.711511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.712955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.726019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.733443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c6733-82b5-4e7e-9614-c2df71164496 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.768050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.768652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.768926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.769191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.769829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.771467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.789145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 369/745] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:06:37 2022] POST /identity/v3/services => generated 251 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:37.790539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.824256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 377/746] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Aug 30 04:06:37.825526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:37.837296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:37.937160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3521479e-b157-45b0-bcc2-6b9eff40f024 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.942313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ce20d7-b959-46de-9c16-b8942f1fcb6e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:37.981525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.981741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:37.982133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.982238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:37.982461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.982557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:37.982722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.982817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:37.983379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.983486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:37.983729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:37.984719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.985115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:37.994813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 370/747] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:37.996050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.002449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 378/748] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:06:37 2022] POST /identity/v3/services => generated 240 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:38.003459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.024831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.100244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.160565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0584eec6-af86-4a78-95c0-c857287a9439 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.199864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.200422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.200696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.200962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.215557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.217021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:38.238176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:38.239692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:38.240718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:38.240718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:38.240718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:38.241974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 379/749] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:38.242863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.279686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347a88f3-d044-408a-bbd5-02c7a72c8314 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.322422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.323681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.324308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.324860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.325781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:38.326181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.327964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.344319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 371/750] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 349 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:38.347195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.384890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e26fda-f07c-41ad-a9ec-455c549ebe39 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.418709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.422855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.423492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.423776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.424047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.436962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.438979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.441014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:38.457436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:38.458659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:38.459714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:38.459714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:38.459714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:38.459714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 380/751] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:38.460854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.480145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c577d40-8500-4e27-9f12-05d6191e7d26 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.527245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.528322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.528796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.529520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.530621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:38.531138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.532976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.548234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 372/752] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:38.549423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.578231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4becdf7-c2fa-46c6-8aff-69a6a909ec9a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.622607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.623264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.623545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.623853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.633235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.635457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:38.650665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:38.652254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:38.653246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:38.655700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 381/753] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:38.656684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.688530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.695716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d2ad0d-03c1-4d1d-87e1-4f53d08e64a8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.736135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.742816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.743429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.743708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.743981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.745009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:38.745357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.747130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.764253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 373/754] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:38.764253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.775429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75a3ee1-7c73-4a9a-8fd5-b7b6646e5c6d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.815527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.816467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.816944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.817404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.825078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.826924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:38.839527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:38.840483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:38.841378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:38.843823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 382/755] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:38.844825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.880105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfd6718-2434-4bfc-95b0-e2bc479ab983 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.920156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:38.920761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:38.921036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:38.921306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:38.922058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:38.922326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:38.923940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:38.941335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 374/756] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:38.942405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:38.944511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:38.961439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b516478f-fcfe-49ab-8dc6-4d1bfbbf1e7d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:38.999410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.000358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.000822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:39.001049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.001505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.012625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.014420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:39.027447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:39.028439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:39.029368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:39.036199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 383/757] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:39.037225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.052803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5eb61d6-0452-4067-901c-bfb50abb6c59 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.089044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.090278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.090644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.090913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.091607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:39.091929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.093339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.111375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 375/758] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:39.112152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.148909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca56d12-3fe7-40f4-ae72-43eff2723cd5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.185671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.186656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.187157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.187646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.188737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=fb30d87ff47b49e39091ef401f10ff59) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:39.189242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.190945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.197732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 384/759] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/role_assignments?user.id=fb30d87ff47b49e39091ef401f10ff59&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:39.199561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.218890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:39.240362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc94506e-6f52-4f93-9e03-90b92555b567 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.264223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:39.294125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.294778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.295055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.295325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.296072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:39.296396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.297837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.318697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 376/760] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:39.326657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.362541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e480b26-a2b3-4ac1-b36f-93c041853428 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.404822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.405487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.405703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.405975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.406763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=f3da3928b5414f14ae49e96282a135f1) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:39.407069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.408727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.415548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 385/761] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/role_assignments?user.id=f3da3928b5414f14ae49e96282a135f1&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:39.544168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87370818-0678-4412-a107-fa09dc63b534 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.581559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.582631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.583159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.583662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.601120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.602996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:39.623488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:39.624533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:39.625431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:39.625431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:39.625431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:39.627626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 377/762] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 301 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:39.633386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.731630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:39.762196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b160623-4593-4736-96f1-9d58347ffab9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:39.816875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:39.817170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.817452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.817725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.818405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:39.818711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:39.820079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a99a9ce-d4ab-47a8-b26d-ba577c32c21a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:39.840948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 386/763] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:39.846398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:39.876676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147aac-8bda-4684-98ce-0d2b4902a8a0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147aac-8bda-4684-98ce-0d2b4902a8a0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.878292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0147aac-8bda-4684-98ce-0d2b4902a8a0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.879954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 387/764] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:39 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:39.884978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:39.885377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:39.885745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:39.898868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:39.920021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:39.926595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:40.022170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa384809-216c-4b13-9dc3-3f60d65bee13 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.033536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 388/765] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:39 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:40.061701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.062360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.062635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.062899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.073322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.075334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:40.094536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:40.095667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:40.096757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 378/766] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:40.103087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.178989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:40.227266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ff83a-7f8d-411a-98b8-3ee3d3a5fe7d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.272156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.272994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.273447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.273859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.274785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:40.275229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.277091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c3656b-7aad-4022-951a-20046a775ed7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.289405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 389/767] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:40.295396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.297399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70345-6e2f-472e-b8a6-f90fbeeb5ef1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.297832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70345-6e2f-472e-b8a6-f90fbeeb5ef1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.298279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb70345-6e2f-472e-b8a6-f90fbeeb5ef1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.299692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 390/768] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:40.304675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.305192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.305607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.325776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:40.327269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:40.395734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 391/769] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:40 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:06:40.430146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.498626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:40.531452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554d5aad-52d9-43f3-808b-05fd5c577c93 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.533137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:40.599253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.599919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.600195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.600462 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.606469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=148e575ef9b642c8b81d7eca8dd70bfa, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.607921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.616918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e8e605-926b-4540-bb33-d2f21cf00e0a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.637934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 379/770] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:40 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 343 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:40.665903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.666574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.666849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.667923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.668335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.669802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:40.679820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:40.680918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.681763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 392/771] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:40.687409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.720300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.863551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4093092-f5fc-4aee-b799-f810e44fd269 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.895036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adfc8904-0a6b-4b34-b33b-1c4ad30b7a35 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:40.906310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.906310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.906561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.906819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.907542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:40.907755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.909307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.918343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 380/772] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:40.921468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:40.934633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:40.938699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:40.940333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:40.941237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:40.941606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:40.941972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:40.942714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:40.944435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:40.954643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:40.955885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:40.956957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:40.958913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 393/773] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/services/network => generated 88 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:40.966992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.108280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8b0309-95a3-4e9f-ba26-de278f588256 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.156870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5bbf88-0521-4508-8597-3e32a2d378ce None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.177083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.177695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.177969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.178231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.178884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:41.179179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.180539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.192720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 381/774] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:41.198906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.220712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.221473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.221750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.222014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.222902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] RBAC: Adding query filter params (name=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:41.224018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.225401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.238653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 394/775] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:41.243526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.323155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:41.333880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:41.385555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c6acc9-9fd3-4de5-a68c-24d306a713eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.427039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3d6ecb-0abe-48df-83f9-486d55f2f787 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.457085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.457709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.457986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.458281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.458989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] RBAC: Adding query filter params (interface=public, service_id=2a928b3bbc094e4d90b4c4290974e54f, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:41.459295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.460669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.475010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 382/776] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:06:41 2022] GET /identity/v3/endpoints?service_id=2a928b3bbc094e4d90b4c4290974e54f&interface=public®ion_id=RegionOne => generated 194 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:41.480522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.503340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.504057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.504248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.504473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.505098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] RBAC: Adding query filter params (type=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:41.505393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.507042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6cbb5f2-8617-47eb-b02c-e6ba0e3f2146 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.518898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 395/777] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:41 2022] GET /identity/v3/services?type=network => generated 350 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:41.520236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ee3e-6831-4694-9638-46301179bf8d None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.520518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ee3e-6831-4694-9638-46301179bf8d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.520788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee91ee3e-6831-4694-9638-46301179bf8d None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.522515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 396/778] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:41.525472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91ee3e-6831-4694-9638-46301179bf8d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.613719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f839a8f9-3a5b-4ac1-8bae-4b40babc6ceb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.656306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.657648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.658574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.659327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.660170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.661864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee91ee3e-6831-4694-9638-46301179bf8d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.662229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c15b1dd-0d8c-472b-a58b-c1aabdb0c0b5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.663923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:41.671622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 383/779] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:06:41 2022] GET /identity/v3/services => generated 772 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:41.673714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d306-a966-4177-97e9-ac053cd0c68f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.674215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d306-a966-4177-97e9-ac053cd0c68f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.675153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7937d306-a966-4177-97e9-ac053cd0c68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.677837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:41.701311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7937d306-a966-4177-97e9-ac053cd0c68f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:41.702577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7937d306-a966-4177-97e9-ac053cd0c68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:41.719260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.720273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.720466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.720734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.721459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] RBAC: Adding query filter params (interface=public, service_id=cb6eced020034d8699982639bace82b5, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:41.721766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.724801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.743753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 397/780] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:06:41 2022] GET /identity/v3/endpoints?service_id=cb6eced020034d8699982639bace82b5&interface=public®ion_id=RegionOne => generated 194 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:41.751516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:41.853277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 384/781] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:41.856763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368d6b-2ff9-4542-9e46-0eacd8c986cc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.857119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368d6b-2ff9-4542-9e46-0eacd8c986cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.857470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40368d6b-2ff9-4542-9e46-0eacd8c986cc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.862279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 385/782] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:41.865050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.865331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.865601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.892466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:41.894133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:41.938093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea851fc-9b5f-4799-9de6-e81bc846693b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:41.976535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:41.977410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:41.977937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:41.978205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:41.978841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:41.980315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:41.986862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 398/783] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:06:41 2022] GET /identity/v3/services => generated 772 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:41.997076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 386/784] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:41 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:42.216479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.260356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:42.323062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6208d1-9926-484b-86fa-83846dd2e30a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:42.368275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.371274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:42.372011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:42.372295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:42.372563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:42.385026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:42.392131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:42.393095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:42.393972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:42.396961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 399/785] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:42.404475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.489034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:42.532918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4946f920-00d4-484e-8603-6034ed2d026f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:42.576839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:42.577526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:42.577804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:42.578070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:42.600051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:42.605551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:42.606518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:42.607372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 387/786] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/roles/service => generated 85 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:42.615675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.638812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:42.643599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83aac067-b383-45c0-9917-fd48efe37349 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:42.738976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:42.739669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:42.739895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:42.740147 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0815ca51-61b4-4fb0-be8a-829b799d2af8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:42.740255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:42.740837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:42.741144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:42.742548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:42.761280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 400/787] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 357 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:42.765954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:42.771479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.777119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:42.777741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:42.778024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:42.778326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:42.779004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:42.779309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:42.780925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:42.790039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 388/788] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:42.796556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:42.916692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e313c68-2211-46e1-af22-d7e6f0bc21fa None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:42.959196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:42.960177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:42.960681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:42.961149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:42.981590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:42.983569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.000522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f47558-faeb-461a-95d5-e81ed52234ad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:43.002273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:43.003185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:43.004095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:43.004095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:43.004095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:43.006161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 389/789] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:43.012031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.046531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.047172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.047454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.047726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.058873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.059433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.060896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-01535b81-c1a0-40e1-a43e-755cb2926508 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:43.079004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:43.080016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:43.080918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:43.080918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:43.080918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:43.080918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 401/790] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:43.081821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8973d-5bc4-4dea-bb3c-9266c67ab2f1 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.082010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8973d-5bc4-4dea-bb3c-9266c67ab2f1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.082301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83c8973d-5bc4-4dea-bb3c-9266c67ab2f1 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.083904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 402/791] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:43.091535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.091816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.092128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.115690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:43.116936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:43.131583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.147052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1175d3a-f7e1-4221-8a44-ef06c555f899 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.188671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.189800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.190737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.191447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.192264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:43.192729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.194110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.202628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 403/792] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:43 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:43.203786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.206812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 390/793] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:43.211974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.311947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4f5090-cbca-4f4b-ac64-24f053939ec8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.348994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.349946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.350450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.350890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.351699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:43.352168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.353696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.367020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 404/794] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:43.375488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.386637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.406654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.419290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85330a96-7258-40a1-a46a-37286aa4cca5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.489240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.489869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.490143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.490437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.509515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.511084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:43.538531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:43.539554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:43.540501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:43.543013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 391/795] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 331 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:43.548861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.578168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a88bf38-af85-4714-aa3a-b09021fbfa61 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.619087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.619835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.620111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.620423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.628874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.630466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:43.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:43.646496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:43.648762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:43.650068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 405/796] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:43.654362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.689865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0458212a-e0be-46d7-aa4b-8baaf1246abd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.723352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.738046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.739407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.739862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.740279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.741613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:43.742068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.744231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.767811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 392/797] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:43.768400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf3b7c6-6535-4042-95c2-dc749a7369a3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.769054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.783471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:43.821426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.822402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.823336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.824543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.839465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:43.847989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:43.849002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:43.849878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:43.853284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 406/798] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:43.854393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:43.903612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b129d7-5948-484b-a13b-89f27cc6dab1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.949937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:43.951542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:43.952035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:43.952926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:43.954177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:43.954710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:43.957373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:43.978216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7edc9c-3f03-47b1-a341-78890b1b5bf8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:43.988584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 393/799] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:43.989559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.008242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.024404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.025554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.026041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.027957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.041490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.043526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:44.057917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:44.059096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:44.060015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:44.060015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.060015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.060015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 407/800] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:44.060243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.068343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.103601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9388cae8-7051-44c7-9e83-03f548956f5d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.140800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.141435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.141713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.142347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.142675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:44.142974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.144339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28092da-27d3-43d9-8c66-27a7075e9dae None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.153239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 394/801] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:44.154458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3c08-6e26-49dc-8a89-572a9b845a3f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.154739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3c08-6e26-49dc-8a89-572a9b845a3f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.155008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07da3c08-6e26-49dc-8a89-572a9b845a3f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.160206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 395/802] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:44 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:44.160960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da3c08-6e26-49dc-8a89-572a9b845a3f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.179126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3be73b0-5f59-4bfb-beb3-a3274e2c5326 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.219479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.220105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.220384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.220650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.231775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.233289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:44.248046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:44.249236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:44.250129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:44.250129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.250129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.250129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 408/803] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:44.250413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.277717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07da3c08-6e26-49dc-8a89-572a9b845a3f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.279253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.309922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.316152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.316761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.317034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.317301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.317937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:44.318237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.319621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2da58f6-c54d-49f6-813d-c8d97636c5cd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.330711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 396/804] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:44.331883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d17d5-e70e-4b33-bc4c-c2efcdba75e6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.332169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d17d5-e70e-4b33-bc4c-c2efcdba75e6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.332441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4d17d5-e70e-4b33-bc4c-c2efcdba75e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.350155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ed4d17d5-e70e-4b33-bc4c-c2efcdba75e6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:44.351375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed4d17d5-e70e-4b33-bc4c-c2efcdba75e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:44.353746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8622da07-c97e-406e-ba6c-ac2b4896579f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.388530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.389151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.389427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.389692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.400276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.402340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:44.415690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:44.416791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:44.418310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:44.418310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:44.418310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.418310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 409/805] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:44.418310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.424227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 397/806] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:44.425497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599558-a853-48b0-a7ff-3eaf3d6ab952 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.425780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599558-a853-48b0-a7ff-3eaf3d6ab952 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.426054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a599558-a853-48b0-a7ff-3eaf3d6ab952 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.427459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 398/807] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:44.428001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a599558-a853-48b0-a7ff-3eaf3d6ab952 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.547492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a599558-a853-48b0-a7ff-3eaf3d6ab952 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.571311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.588705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.589529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.590285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.590285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.598205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.599739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:44.612678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:44.613612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 399/808] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:44.619587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.630747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.655081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba7ece-67c2-4870-96ef-0f00ace241e6 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.690918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.691524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.691786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.692205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.692676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:44.692972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.694349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e8432bd-6bc9-401f-b72b-f64db02a8e60 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.706862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 410/809] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:44.708083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.708364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.708632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.726479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:44.727675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:44.735793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3414d0e3-3a6d-4189-ab8e-0203e15e363f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.770040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.770677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.770954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.771477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.772114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:44.772415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.773761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.785740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 400/810] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:44.786634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.797175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 411/811] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:44 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:44.798020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.835313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.874759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:44.904397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e475c56f-e560-40eb-90e6-87b1ead88dd6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.906744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5520ff-bef9-4833-adcc-1f7fddac9fe5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:44.941793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.942003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:44.942453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.942561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:44.942757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.942873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:44.943026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.943122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:44.943602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.943698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:44.943997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:44.944958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.945557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:44.954427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:44.955459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:44.959344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 412/812] 10.222.0.46 () {56 vars in 1129 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:44.960805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:44.969884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 401/813] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:44.969884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.065788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ef763-d5fb-418b-9bac-b47917064ce2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.069228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.072267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cc4d3b-6738-4a05-973c-611d7c177ee2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.099721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.100339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.100612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.100881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.102746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.107762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.108388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.108679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.109053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.109344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.110976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.116583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.118011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:45.123171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:45.124225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 413/814] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:45.130151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:45.130849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:45.132110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 402/815] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:06:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:45.133345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.231307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6701c7-adf3-4ece-8f94-19879c4f4add None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.231569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54712f92-e29d-494b-b60d-281a5511e13c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.265779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.266151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.266459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.266733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.266824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.266999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/f3da3928b5414f14ae49e96282a135f1/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.267101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.267298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.267916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:45.268216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.269565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.274153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 414/816] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:45.274961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.285965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=f3da3928b5414f14ae49e96282a135f1, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.287444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.300035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 403/817] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:45 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/f3da3928b5414f14ae49e96282a135f1/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:45.300825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.315092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.345991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.399032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f2e4a-b559-40de-9a24-92e0a207f247 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.415912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7eb155-0d6f-4fbe-8551-20ef9e58a747 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.440273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.440908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.441189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.441455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] PATH_INFO: `/v3/services/network` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.442302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.444440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:45.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:45.459524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:45.460382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 415/818] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/services/network => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:45.461202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.469237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.469866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.470146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.470451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.471192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:45.471473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.472914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.486474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 404/819] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:45.487232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.565218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.578265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8045a3d1-270e-43bf-bb11-79363102b2d7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.601126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.607334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a059932c-0235-4a68-bfc9-fa8ef8f2180e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.614910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.615528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.615808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.616072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.617183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:45.617487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.619367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.639872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 416/820] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:45.644842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.649402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.649949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.650347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.650600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.651171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:45.652147 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.653657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.659331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 405/821] 10.222.0.46 () {56 vars in 1124 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:45.660003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.756629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d46fd5da-c732-4b0a-84dc-8a8915a5e1d3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.769846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a3fd3-f835-4452-8ee6-46844106c26d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.795011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.795628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.795927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.796177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.796816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] RBAC: Adding query filter params (name=network) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:45.797132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.798503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.803356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 417/822] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/services?name=network => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:45.805583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:45.806261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:45.809702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:45.809702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/fb30d87ff47b49e39091ef401f10ff59/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:45.809923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.814789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=fb30d87ff47b49e39091ef401f10ff59, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:45.816238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:45.818782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.838348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 406/823] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:06:45 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/fb30d87ff47b49e39091ef401f10ff59/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:06:45.838348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:45.875351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:45.957565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d26c96f-e111-4f9c-b6e2-27596278cb5a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:45.975012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3825a5d2-26a1-4181-bc4f-2dbae3e3c87f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.005763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.006682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.007104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.007540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.027252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.027957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.028175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.028444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.029139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.030835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.056501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 407/824] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:06:45 2022] POST /identity/v3/endpoints => generated 346 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:46.058083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.097136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.098858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.151855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:46.187497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:46.219600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 418/825] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:45 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:06:46.225051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.310067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e6cdf4-427c-41cd-9f49-2cfc876f3228 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.352426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.353327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.353752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.354165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.354866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2a209-1e59-48d1-a118-afc76aff15e5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.355192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] RBAC: Adding query filter params (type=network) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:46.355663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.357186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.363782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 408/826] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:06:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:46.371002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.398134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.399054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.399813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.400177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.415079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.418945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.427329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:46.444470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:46.445645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:46.447086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:46.447086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:46.447086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:46.449660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 419/827] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:46.455501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.502371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f41febb-8e81-4f63-8dae-48dd6ca50a9f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.514497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:46.553772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.554977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.555425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.555853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.556931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.559747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.584718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 409/828] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:06:46 2022] POST /identity/v3/endpoints => generated 337 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:46.585564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a78462e3-b1db-4f3f-be19-fce770cf2c6e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.637936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.638576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.638848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.639366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.640193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:46.640590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.642098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.663847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 420/829] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:46.669692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.812683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7bb3f53-b419-4e79-a408-ec4c69407d2d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:46.850835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:46.851845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:46.852330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:46.852819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:46.861234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:46.862986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:46.875054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:46.876067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:46.876961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:46.879355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 410/830] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:46.885266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:46.943183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:47.004780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185a2bdd-c2fc-4ea8-a70d-c4d37e5c517b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:47.043511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:47.044145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:47.044426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:47.044699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:47.045401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:47.045707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:47.047156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b7b7f27-794a-494c-be57-91227158936b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:47.064617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 421/831] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:47.070076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:47.110612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:47.200405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a806601-3c05-49cb-91c9-d47990febdcb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:47.236369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:47.236978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:47.237317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:47.237590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:47.238341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:47.238655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:47.240078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c306ff50-239d-4896-a1fd-d5db1f8a50e6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:47.246517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 411/832] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:47 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:47.506726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e93dd0-fc03-421f-99be-b2e63c59d0b8 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:47.507086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e93dd0-fc03-421f-99be-b2e63c59d0b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:47.507394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e93dd0-fc03-421f-99be-b2e63c59d0b8 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:47.509014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 422/833] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:47.515298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb98efd-3d3c-4e1f-8f08-b8041703a2fe None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:47.515580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb98efd-3d3c-4e1f-8f08-b8041703a2fe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:47.515858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eb98efd-3d3c-4e1f-8f08-b8041703a2fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:47.536713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7eb98efd-3d3c-4e1f-8f08-b8041703a2fe None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:47.537953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7eb98efd-3d3c-4e1f-8f08-b8041703a2fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:47.619028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 412/834] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:47 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:47.918371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e93dd0-fc03-421f-99be-b2e63c59d0b8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.040874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e93dd0-fc03-421f-99be-b2e63c59d0b8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.049216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244bba-6c91-4386-895f-4d328b397fe8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.049707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244bba-6c91-4386-895f-4d328b397fe8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.049788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb244bba-6c91-4386-895f-4d328b397fe8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.051559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 413/835] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:48.056624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.057015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.057391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.058650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.081483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:48.083076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:48.083291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.083932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.084211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.084482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.098707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:48.105250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:48.106294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:48.111377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 423/836] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:06:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:48.113893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.114612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.178296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 414/837] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:06:48.232380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88720d81-b099-4184-ac9e-67ddc32bab6b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.268417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.269030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.269312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.269583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.270275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:48.270582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:48.272277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3e2a58-dfef-4413-91ec-e0d8739dd1d2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:48.282032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 424/838] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:48.287661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.385736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244b09-2991-431d-8c6a-e4ac7b6917b9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.385923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244b09-2991-431d-8c6a-e4ac7b6917b9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.386224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02244b09-2991-431d-8c6a-e4ac7b6917b9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.387935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 425/839] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:06:48.393639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.393956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.394184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.400163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de3127e-9f61-4f04-917f-c06d65e632a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.414236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:48.415470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:48.443754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.444728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.445199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.445743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.450991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.459193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:48.461017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:48.478221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:48.479296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:48.480380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:48.480380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:48.480380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:48.480380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 415/840] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:48.481051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.485289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.503977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 426/841] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:48 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:06:48.510466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.618374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be33a5d-a976-4eeb-8d31-d6a21697a6eb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.624865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b249e4-c62e-41dc-9c76-7a99b39aecbc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.657485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.658077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.658416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.658690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.671447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:48.674000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.674860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.675365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.675885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.676729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:48.677111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:48.678106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:06:48.678989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:48.680440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:48.682021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:48.684254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 416/842] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:48.685118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.703901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 427/843] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:48.705592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.742533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.758855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:48.804122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07966c30-edd7-4c3c-9634-2d5db16c2512 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.840062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b397b4-3bb2-4463-9af0-78e4fc01ed6f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:48.841347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.842025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.842327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.843388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.843388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:48.843599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:48.844894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:48.854910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 417/844] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:48.855522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.883991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:48.884495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:48.884774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:48.885044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:48.893558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:48.895023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:48.907470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:48.908508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 428/845] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:48.910046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:48.995816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981d569e-0310-457a-b44d-e3cacebd7352 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.006417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.025092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd78035-2983-476a-9d4a-7a6e0c193534 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.026757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.039961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.040601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.040880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.041153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.041736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.043294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:49.052634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:49.054374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:49.055310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 418/846] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:49.056138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.067127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.067751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.068030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.068327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.080280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.081707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:49.096207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:49.097430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:49.098824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:49.098824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:49.098824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:49.102111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 429/847] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:49.102976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.165937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef8f10c-13cf-4424-b3ed-756c06d2e51f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.203660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.204436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.204838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.205208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.206051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:49.206492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.208033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.214704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246d4722-da9e-4edd-a5c7-3ee5d0028128 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.227707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 419/848] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:49.228478 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.255348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.255955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.256236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.256511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.257134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:49.257438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.258848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.260745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.269251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 430/849] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:49.269251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.283808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.344697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f5f52b-f0ff-484e-804e-53477baf7919 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.379117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.379848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.380125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.380393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.380957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c711db-9b2c-4a7f-b973-c2ede9d62199 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.381057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:49.381314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.382697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fca2df-eb53-413e-b3d5-41d90fb03ed1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.394552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 420/850] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:49.395852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93590-809f-4d2c-aa29-bc6dc6f4aed3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.396402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93590-809f-4d2c-aa29-bc6dc6f4aed3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.396672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f93590-809f-4d2c-aa29-bc6dc6f4aed3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.401005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 421/851] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:49.401005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93590-809f-4d2c-aa29-bc6dc6f4aed3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.419468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.419998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.420277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.420541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.485432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.487983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.489710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.503393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f93590-809f-4d2c-aa29-bc6dc6f4aed3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.510395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.538427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.539023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.539293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.539555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.540172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:49.540469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.542392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ea14586-f2ad-46fe-81eb-c5ee51b72bfe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.546813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 422/852] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:49.547951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.548230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.548500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.569742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:49.571128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:49.599109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 431/853] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:06:49.599109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.654814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 423/854] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:49 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:49.655703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.726011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.733821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba6aac0-8a3d-4a10-9042-13dd97a6e077 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.782085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.783162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.783683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.784151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] PATH_INFO: `/v3/users/swift` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.792614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.793908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:49.795700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.797029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90a332f-1607-410d-b6d0-1e50bd5851d1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:49.821295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:49.822266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Aug 30 04:06:49.823824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:49.826487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 432/855] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:49.832214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.836203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.837659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.837949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.838224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.851159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:49.853169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:49.874528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:49.875552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:49.876456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:49.876456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:49.876456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:49.878330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 424/856] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:49.881171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:49.960185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c64da5-76e0-440b-b21b-461a67b761d7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.996918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:49.997367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307515eb-2429-463a-8463-6bb19f49d24e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:49.997481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:49.997710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:49.997983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:49.998758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:49.999062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.000485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.022292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 433/857] 10.222.0.46 () {56 vars in 1135 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:50.022292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.041157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.046174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.046812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.047162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.047353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.047991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.048287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.049634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.065761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 425/858] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:50.065761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.080974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.132726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603088a9-c82b-4561-a2d7-33e894f9d053 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.169390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.170015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.170315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.170591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.171180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.172538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.174719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f72cdf-b467-4f82-8227-ee44514c8a05 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:50.183742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:50.184877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:50.186048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 434/859] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:50.187047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.215988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.216711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.216985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.217267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.269680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.288381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2ce6a3-5be7-47a8-b569-3c8ec96aad79 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.291205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.293206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.308604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.324174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.324890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.325237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.325577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.333861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:50.349525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:50.350772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:50.351712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:50.357159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 435/860] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:50.357911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.410328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 426/861] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/auth/tokens => generated 1161 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:06:50.411509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.461489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c424d-1c16-4b86-ad7b-aa0760af1c15 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.497882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.498510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.498792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.499041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.499660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.499966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.501288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.510125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 436/862] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:50.510125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.519073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.520704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e886fb95-0c25-461f-a011-ec2190623279 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.536616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.562628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.563301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.563571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.563839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.564527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.564828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.566382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.583163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 427/863] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:50.583163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.626896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb0bb0b-b191-44b8-bd04-f873aa4edf29 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.680794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.681783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.682221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.683301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.699954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.702858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:50.725654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:50.727588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:50.730191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:50.730191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:50.730191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:50.733781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 437/864] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:50.734723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.735719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ffa95ae-3638-4929-8ef8-d53445c5a59c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.774624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.775148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.775425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.775691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.776848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.777144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.778649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.783204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 428/865] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:50.783956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:50.813267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.840837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:50.867676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21fdb774-5b7f-401b-93c7-587eb26304bc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.916581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.918476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.918935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.919633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.921091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=f3da3928b5414f14ae49e96282a135f1) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.921863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.924291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:50.931550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa88e294-565f-4626-a13d-658eb349c8c4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:50.938568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 438/866] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/role_assignments?user.id=f3da3928b5414f14ae49e96282a135f1&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:50.980948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:50.981581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:50.981865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:50.982141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:50.982865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:50.983172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:50.984605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:51.002992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 429/867] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:51.008880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:51.127436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0894856-4a0d-4e54-9d1f-50cfc0bdcb06 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:51.154100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:51.169039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:51.169678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:51.169951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:51.170216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:51.179301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:51.180721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:51.193302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:51.194293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:51.195225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 439/868] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:51.202058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:51.310751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:51.317989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-869737ad-78d8-4cb6-8ff6-015adba4e8be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:51.356565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:51.357242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:51.357522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:51.357793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:51.358582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:51.358937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:51.360411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5256b203-bcc6-4eb1-96a0-875792a9a67e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:51.373456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 430/869] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:51.379150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:51.487139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e4de48d-dd92-48bd-9602-e14b23bff1ee None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:51.522619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:51.523221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:51.523491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:51.523763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:51.524484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=fb30d87ff47b49e39091ef401f10ff59) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:51.524785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:51.526207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152fc8af-b018-4963-85d9-9a193df11248 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:51.533450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 440/870] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:51 2022] GET /identity/v3/role_assignments?user.id=fb30d87ff47b49e39091ef401f10ff59&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:51.578480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab762d-6f01-4990-9870-1a28dc727b56 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:51.578880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab762d-6f01-4990-9870-1a28dc727b56 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:51.579241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab762d-6f01-4990-9870-1a28dc727b56 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:51.581034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 431/871] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:51.586810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe927-2aab-44ce-a78c-8cad03e5e82b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:51.587097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe927-2aab-44ce-a78c-8cad03e5e82b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:51.587525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66dfe927-2aab-44ce-a78c-8cad03e5e82b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:51.607815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-66dfe927-2aab-44ce-a78c-8cad03e5e82b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:51.609149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66dfe927-2aab-44ce-a78c-8cad03e5e82b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:51.661848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:51.701349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 441/872] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:51 2022] POST /identity/v3/auth/tokens => generated 1161 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:51.985265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab762d-6f01-4990-9870-1a28dc727b56 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:52.112177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab762d-6f01-4990-9870-1a28dc727b56 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:52.154516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:52.155126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:52.155491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:52.155777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:52.156492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:52.157872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405bbabb-fa55-4043-bd4f-36765fc87035 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:52.174086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 432/873] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:06:51 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:52.825604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7566-b4fd-4623-a1dd-5b78cc54f3a3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:52.826139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7566-b4fd-4623-a1dd-5b78cc54f3a3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:52.826139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f7566-b4fd-4623-a1dd-5b78cc54f3a3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:52.827709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 442/874] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:52.833061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607129-1826-40b1-bd77-f219cfed3402 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:52.833461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607129-1826-40b1-bd77-f219cfed3402 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:52.833650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607129-1826-40b1-bd77-f219cfed3402 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:52.851691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b1607129-1826-40b1-bd77-f219cfed3402 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:52.852865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b1607129-1826-40b1-bd77-f219cfed3402 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:52.863724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:52.929269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 433/875] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:52 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:53.159332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfcb01-de71-4d72-b084-8d24cc2d0135 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.159744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfcb01-de71-4d72-b084-8d24cc2d0135 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.159925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cfcb01-de71-4d72-b084-8d24cc2d0135 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.161452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 443/876] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:53.166591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.167360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.167993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.186657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:53.188180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:53.198585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfcb01-de71-4d72-b084-8d24cc2d0135 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:53.275178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 434/877] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:53 2022] POST /identity/v3/auth/tokens => generated 1262 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:53.327095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cfcb01-de71-4d72-b084-8d24cc2d0135 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:53.365913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:53.366575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.366860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.367136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.367827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:53.369283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9d129b-4ecb-48ce-b275-c00e7157f194 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:53.387657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 444/878] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:06:53 2022] POST /identity/v3/services => generated 269 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:06:53.553579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:53.588859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74487c1a-e20d-4b0f-a0fa-32990f9e97cd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.589540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74487c1a-e20d-4b0f-a0fa-32990f9e97cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.590038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74487c1a-e20d-4b0f-a0fa-32990f9e97cd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.591741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 445/879] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:53.599515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.599815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.600201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.618938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:53.620854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:53.628020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:53.656618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:53.672021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d652db32-f605-49d0-a16c-f6ce65643700 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:53.717559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:53.718195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.718502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.718772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.730472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 446/880] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:53 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:06:53.737181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:53.742345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:53.743508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:06:53.744560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:53.747767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 435/881] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:06:53 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:53.754283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:53.863438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2917f8cf-68aa-4276-af15-586b24f0ea6f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:53.904657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:53.905271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:53.905549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:53.905869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:53.906560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:53.906863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:53.908230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:53.917768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 447/882] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:06:53 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:53.923295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.029747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e20677-24cb-4d54-a415-3e163536e5af None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.041501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.062375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.071499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.072112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.072391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.072715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.076234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.085788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.087474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:54.107078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:54.108101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:54.109007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:54.109007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:54.109007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:54.115719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 436/883] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:54.117586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.177542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e9f289-34d5-4e9f-a65f-6e9a9d38e6e2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.214616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.215502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.215952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.216366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.217170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.218720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:54.228171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:54.229798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:54.230809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.231672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c70eb-f010-4a20-9766-b7b9d60dfbe3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.233843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 448/884] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/services/object-store => generated 93 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:54.241093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.277140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.277862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.278142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.278454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.279109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:54.279406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.280781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe3cdd6-3b3f-4633-843d-9604ceba3a69 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.293429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 437/885] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:54.299317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114915-1869-45af-a8f8-95338cbb2103 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.299602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114915-1869-45af-a8f8-95338cbb2103 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.299883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea114915-1869-45af-a8f8-95338cbb2103 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.301104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 438/886] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:54.301793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea114915-1869-45af-a8f8-95338cbb2103 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.345916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.351467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.365840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e488b121-9b20-42ed-8529-e1bd8cf0c2d3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.400143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.400745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.401042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.401960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.402615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:54.402917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.404274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dbf11ab-49bd-4c30-9037-d49ad2a9d4fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.408978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 449/887] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:54.409636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea114915-1869-45af-a8f8-95338cbb2103 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.410183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.410498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.410770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.430593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:54.432032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:54.451822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.452469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.452750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.453018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.464857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.466372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:54.479653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:54.480690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:54.481658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:54.485933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 439/888] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:54.486885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.504447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 450/889] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:54 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:54.505685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.606636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.616437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1c6ff3-83fb-4357-a8f9-24b1ee79591b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.618626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.630365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567a2c76-7025-46a8-b9bf-cee661e99077 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.656333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.657246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.657784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.658228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.659167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:54.660184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.661994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.670037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 440/890] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:54.682304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.682304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.682304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.682304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.682741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:54.682741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.683765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.705810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 451/891] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:54.711291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.801167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:54.830553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94811c6-5f19-417e-a544-ba41281d1aee None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.882367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.882913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.883185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.883455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.924068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:54.933653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35cded01-403c-4ec1-ae39-06e5504c2d5d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:54.965973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.967460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.971306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:54.972189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:54.972693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:54.973151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:54.973936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:54.975505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:54.987192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:54.988294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:54.989121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:54.997203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 452/892] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:54.998464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.002862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.082342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 441/893] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/auth/tokens => generated 1372 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:06:55.089363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.103780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452127a3-fe56-4b80-b6ab-11c20ce0651b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.140077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.140917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.141337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.141737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.142533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:55.142997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.144507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.150161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 453/894] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:54 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:55.157754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.163620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.207106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e0f7ca-c966-4e3e-9b4c-55981d868d2d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.246353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.246563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.247616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.248843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.248843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.260660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.262802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:55.279356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:55.280335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:55.281246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:55.281246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:55.281246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:55.287585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c22e31-190a-4662-80c3-699f82cb80a1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.287975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 442/895] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:55.289033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.321121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.321733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.322012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.322304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.322995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:55.323296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.324657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.334817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 454/896] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:55.335919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.400501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf239e53-8790-48f4-bf27-b5f5550a02c6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.418319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.438428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.439431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.439906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.440414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.441250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:55.441365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8d087cb-a4e0-47ba-8a15-f4ef63d4c09e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.441883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.443464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43c09efb-6180-44bf-8b24-cbc9ecaa25f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.456850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 443/897] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:55.458426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f0a82-1894-4d6c-a87f-179bda5ecd19 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.458911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f0a82-1894-4d6c-a87f-179bda5ecd19 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.459399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280f0a82-1894-4d6c-a87f-179bda5ecd19 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.461053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 444/898] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:55.462095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f0a82-1894-4d6c-a87f-179bda5ecd19 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.474383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.480325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.481377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.481656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.481924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.482697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] RBAC: Adding query filter params (interface=public, service_id=56776b148c4d44af9f223f68149a5711, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:55.483003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.484365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b2198d0-f900-485d-9054-fc15df485e61 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.489878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 455/899] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/endpoints?service_id=56776b148c4d44af9f223f68149a5711&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:55.491163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.491443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.491711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.509385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:55.511141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:55.579704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280f0a82-1894-4d6c-a87f-179bda5ecd19 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.582348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 456/900] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:55 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:55.582348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.623503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.623503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.623819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.623989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.630912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.634575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:06:55.648370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:55.649322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:06:55.650298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:55.654885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 445/901] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:55.656449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.694459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.704862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c59b230b-e2a6-4004-9604-daca82a8452b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.743310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.754782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.756148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.756868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.757292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.758343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.761176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.768941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfe9ef-ddda-435b-8ac4-42da08aae2d8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.774686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 457/902] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/services => generated 1268 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:06:55.820435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:55.821062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:55.821337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:55.821604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:55.822460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:55.822764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:55.824291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:55.838487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 446/903] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:55.845439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.856180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:55.968809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5555c9-ea44-449d-83f3-8e5bcad3c53e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:55.970183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:55.979276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccf60a77-b158-4655-be90-deae1d910ecb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:56.017561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.018640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.019180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.019575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.019839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.020334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.020682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.021352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.021547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] RBAC: Adding query filter params (role.id=14e1f67917c94aee9b0c6c78e38790de, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=f3da3928b5414f14ae49e96282a135f1) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:56.022029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.022130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.023581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.023938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.031874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 458/904] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/role_assignments?user.id=f3da3928b5414f14ae49e96282a135f1&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=14e1f67917c94aee9b0c6c78e38790de => generated 259 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:56.033859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:56.034807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:56.035720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 447/905] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 30 04:06:55 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:56.041741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:56.110523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:56.165311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5f60ca-c7ca-46a2-b5bb-74fd03ce1cd9 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:56.205552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.206168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.206478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.206750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.207383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:56.207686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.209048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.215118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 459/906] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:56.220811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:56.329895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997ed2ab-3fd6-4f01-8237-feb840788263 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:56.365093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.365759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.366041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.366351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.367040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:56.367349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.368715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.374986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 448/907] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:56 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:56.383424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:56.491889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca27ab2a-72a1-40dd-8ff1-9d9ea53203b3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:56.510679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:56.529918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.530561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.530836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.531103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.531806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] RBAC: Adding query filter params (interface=public, service_id=b3d0b07f466846a18fd043e7fa423564, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:56.532180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.533536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9587491-41ee-4282-81d6-23b9ced6cfc4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.539405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 460/908] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:06:56 2022] GET /identity/v3/endpoints?service_id=b3d0b07f466846a18fd043e7fa423564&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:56.546138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:56.573721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:56.654450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411961f9-deb4-4652-a511-c7010b060964 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:56.693493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:56.694100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.694406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.694674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.695275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:56.696964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:56.698657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c104-5033-412c-b7fa-d5c9d1c59e07 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.698939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c104-5033-412c-b7fa-d5c9d1c59e07 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.699210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a5c104-5033-412c-b7fa-d5c9d1c59e07 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.701344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 461/909] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:56.707227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94948-1627-4d01-a47b-31bf7778e2c5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:56.707624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94948-1627-4d01-a47b-31bf7778e2c5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:56.707987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce94948-1627-4d01-a47b-31bf7778e2c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:56.709234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 449/910] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:06:56 2022] GET /identity/v3/services => generated 1268 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:06:56.732322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0ce94948-1627-4d01-a47b-31bf7778e2c5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:56.733801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0ce94948-1627-4d01-a47b-31bf7778e2c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:56.810508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 462/911] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:56 2022] POST /identity/v3/auth/tokens => generated 1372 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:57.086558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:57.190831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:57.196913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f5787b3-5850-472d-a231-eba90fbdecd3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:57.233989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:57.234626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:57.234903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:57.235168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:57.235825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:57.237371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17eec538-859e-4ffd-acc1-f807a1fa75fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:57.254142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 450/912] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:06:57 2022] POST /identity/v3/services => generated 241 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:57.928869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7e01e-e238-4a25-ae84-cb87c6a34384 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:57.929388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7e01e-e238-4a25-ae84-cb87c6a34384 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:57.929388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e7e01e-e238-4a25-ae84-cb87c6a34384 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:57.931034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 463/913] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:57.936548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bfe04-9910-4c13-a749-a190f2650cdd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:57.936837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bfe04-9910-4c13-a749-a190f2650cdd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:57.937112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5bfe04-9910-4c13-a749-a190f2650cdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:57.956373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ed5bfe04-9910-4c13-a749-a190f2650cdd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:57.957565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed5bfe04-9910-4c13-a749-a190f2650cdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:58.037857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 451/914] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:57 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:58.043401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb437f24-aed6-40d9-9d3d-d1e99e2f40c4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.043689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb437f24-aed6-40d9-9d3d-d1e99e2f40c4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.043964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb437f24-aed6-40d9-9d3d-d1e99e2f40c4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.046664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 464/915] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:58.050906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.051197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.051477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.070940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:58.072117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:58.146713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 452/916] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:58.335784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb437f24-aed6-40d9-9d3d-d1e99e2f40c4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.419814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.427687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:58.453492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb437f24-aed6-40d9-9d3d-d1e99e2f40c4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.472672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:58.500943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.501590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.502284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.502567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.503160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:58.505023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:58.517577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:58.518829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:58.520158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 465/917] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:58.524957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.565731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2425778d-9ecb-422e-8070-f46af5f20abc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.602164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.602737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.603007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.603276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.614640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:06:58.619500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:58.620734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:58.627430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 453/918] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:58.631084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.663525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-747c5d21-3b72-4f86-af27-441bba7cf0f2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.703634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:58.706566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.707449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.708206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.708660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.710774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:58.711265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:58.713566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:58.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 466/919] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:58.727065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.751428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d417216-cc58-488d-be19-9f5864dea57b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.780382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:58.807073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.808643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.809150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.809623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.810530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:58.811046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:58.812667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3452afc-aba5-44a7-8cf8-3128b19699ac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:58.826134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 454/920] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:58.827862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f689e-402c-4c6f-a107-767248c0c910 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.828350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f689e-402c-4c6f-a107-767248c0c910 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.829434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6f689e-402c-4c6f-a107-767248c0c910 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.831198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 455/921] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:58.832130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f689e-402c-4c6f-a107-767248c0c910 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:58.856185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-539f277d-5d5b-43dc-b9db-f67936a4fe75 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.896821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.897428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.897707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.897979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.898652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:58.898955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:58.900336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2945f79-5197-417a-be31-335ba8f60e88 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:58.906690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 467/922] 10.222.0.46 () {56 vars in 1114 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:58.912529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.912818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.913091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:58.931654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:58.932979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:58.946184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6f689e-402c-4c6f-a107-767248c0c910 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:58.974798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:58.987888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:58.988518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:58.988808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:58.989070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.000442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.001900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.017182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 468/923] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:58 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:59.017949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:59.018895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:59.019801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:59.019801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:59.019801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:59.019975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.020046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 456/924] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:59.025989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.074806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.146342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404870a8-4761-43e4-84b3-c1bcd8041248 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.158074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9ef8534-5bb7-4a7d-ad13-62a7510fbd3d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.185698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.186423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.186619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.186893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.187685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:59.187877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.189253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.207168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 457/925] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:59.208167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.215296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.216121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.216577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.216997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.218408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.219925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9080bcb8-f98a-407c-b219-7429ba921d95 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.250418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 469/926] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:06:59 2022] POST /identity/v3/endpoints => generated 346 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:06:59.253386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.258758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a20783-a602-4cb2-b391-299dd053d363 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.258758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a20783-a602-4cb2-b391-299dd053d363 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.258758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a20783-a602-4cb2-b391-299dd053d363 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.258758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 470/927] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:06:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:06:59.265613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.265889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.266165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.287225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:06:59.288461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:06:59.335885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efc2776a-1f5d-46f2-9db0-4dc05fa2b7be None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.373036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 471/928] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:06:59 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:06:59.374754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.375416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.375700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.375973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.379239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.386635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.388266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:06:59.403151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:59.404157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:06:59.405107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:59.409521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 458/929] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:59.410790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.411136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.517942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bc7ef8-29a6-4227-9e85-f6b39584c312 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.533191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.537536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9943bb-787f-4a30-b6c7-564287b70645 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.556248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.556886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.557195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.558037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.558660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.560024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:59.569304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:59.570296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Aug 30 04:06:59.571291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.574436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 472/930] 10.222.0.46 () {56 vars in 1132 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:59.576816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.580230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.582063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.582576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.583231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.584185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:59.584714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.587004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.616353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 459/931] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:06:59.618635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.687018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.709058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e668d9fe-67af-48dc-b779-e9eb2bf31d5b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.745077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.745691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.745963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.746229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.746881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:59.747180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.748601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.748707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1476ad-56d8-412b-837c-342204eab3d3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.758206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 473/932] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:06:59.758206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.795254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.795961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.796241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.796516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.814138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.815932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:06:59.833138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:59.834240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:06:59.835351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:06:59.835351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:06:59.835351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:06:59.835351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 460/933] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:06:59.835351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.849129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.866349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75d26e1-896a-4f20-88cc-8896b8005a6c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.906199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.907020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.907213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.907478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.908055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.909396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:06:59.919652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:06:59.920984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:06:59.922042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:06:59.925641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 474/934] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:06:59.926331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:06:59.939159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:06:59.950326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266d4-2d87-4121-991c-9ec09dbec1f6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:06:59.987171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:06:59.987810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:06:59.988088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:06:59.988361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:06:59.989001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:06:59.989303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:06:59.990713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:06:59.996172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 461/935] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:00.001681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.037179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5eb87a-6854-45fa-a034-8815e8f24234 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.074591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.075470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.075790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.076064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.076707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:00.077031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.078052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.078408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.096195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 475/936] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:06:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:00.096195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.115134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df46671d-e878-43b2-89c9-4585e7ffc0f9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.155471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.156080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.156363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.156633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.157265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:00.157566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.158984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.163956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 462/937] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:00.164796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.174135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.212064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dcc883-522e-40f8-b9a5-fce1aece3b00 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.249280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.249885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.250161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.250453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.251113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.252450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.274745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 476/938] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:07:00 2022] POST /identity/v3/endpoints => generated 347 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:00.275418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.288620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f16bf6b-99c3-40d0-a703-d6cab09281f9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.339121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.345989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.346798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.347181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.347523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.356536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.358337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:00.372509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:00.373614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:00.374566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:00.377112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 463/939] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:00.391315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.406340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a98e7b0-184e-47ed-9d65-a7bd19bee6dd None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.436257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.455496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.456110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.456385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.456648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.457307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:00.457605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.459034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.470408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 477/940] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:00.472908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.544318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3717d957-6104-4ca7-87d4-857537478749 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.585453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.586327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.586617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.587067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.588207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:00.589053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.591095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.595284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449bb29c-bb6b-48a4-a24a-1f33e240788a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.613782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 464/941] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:00.616839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.638004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.639062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.639569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.640091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.640988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] RBAC: Adding query filter params (interface=public, service_id=298d08872bc748ed85f41cf51776d6d8, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:00.641499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.643069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.656273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 478/942] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/endpoints?service_id=298d08872bc748ed85f41cf51776d6d8&interface=public®ion_id=RegionOne => generated 194 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:00.657460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:00.690634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.721063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:00.757527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad46477-bb7a-40fe-9dae-bb3345db06db None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.776880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5813bbe-7e21-4b8d-af02-5ba5687e726d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:00.806948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.808113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.808535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.809313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/f3da3928b5414f14ae49e96282a135f1/roles/14e1f67917c94aee9b0c6c78e38790de` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.818870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=f3da3928b5414f14ae49e96282a135f1, role_id=14e1f67917c94aee9b0c6c78e38790de)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.818870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05a0105-14b4-47cf-9371-f87c0d82fe18 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.825273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:00.826021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:00.826330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:00.826625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:00.827331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:00.828691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12581c74-6fcf-478b-8164-d94b56ea9107 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:00.838418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 479/944] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:07:00 2022] GET /identity/v3/services => generated 1497 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:07:00.839201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 465/944] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:07:00 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/f3da3928b5414f14ae49e96282a135f1/roles/14e1f67917c94aee9b0c6c78e38790de => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:07:01.293310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d16ba0-bff6-405c-aee7-bf07d25c922b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:01.293868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d16ba0-bff6-405c-aee7-bf07d25c922b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:01.293868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d16ba0-bff6-405c-aee7-bf07d25c922b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:01.295461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 480/945] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:01.300886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:01.301289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:01.301479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:01.321564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:01.322909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:01.398793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 466/946] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:01 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:01.666405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d16ba0-bff6-405c-aee7-bf07d25c922b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:01.777604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d16ba0-bff6-405c-aee7-bf07d25c922b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:01.794711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:01.820421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:01.821047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:01.821331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:01.821608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:01.822285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:01.824196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:01.827023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING keystone.server.flask.common [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 04:07:01.848264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.sql.core [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 04:07:01.848264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 04:07:01.848264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [parameters: {'id': 'bdf809b20dc44fa4a304550d97fc8f3a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 04:07:01.848264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81502) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bdf809b20dc44fa4a304550d97fc8f3a.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bdf809b20dc44fa4a304550d97fc8f3a. Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:07:01.866200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Aug 30 04:07:01.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return project_ref.to_dict() Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application yield resource Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise self.value Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application session.commit() Aug 30 04:07:01.868081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.session.flush() Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application compat.raise_( Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 04:07:01.868958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application c = connection._execute_20( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 04:07:01.871440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application result.read() Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:07:01.872795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [parameters: {'id': 'bdf809b20dc44fa4a304550d97fc8f3a', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:01.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:01.878820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.Conflict( Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id bdf809b20dc44fa4a304550d97fc8f3a. Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:01.879747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 481/947] 10.222.0.46 () {60 vars in 1140 bytes} [Tue Aug 30 04:07:01 2022] POST /identity/v3/projects => generated 262 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:01.880352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:01.922915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:01.983731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562f4d02-2d13-4b74-88b8-eb538c7b5fea None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:02.024007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:02.024649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.024926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.025196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.032986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:02.034514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-976e85e7-7963-4e28-a977-c081b25c5756 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:02.046085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:02.047078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:02.047978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 467/948] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:02.055077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:02.167226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1d3568-d9bd-41e6-8346-9b033679da5d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:02.205458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:02.206433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.206938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.207408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.208272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:02.208766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:02.210591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5699083-ac69-436a-afb1-4523034c0cb6 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:02.233656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 482/949] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:02 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:02.352877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4687-6f4e-4dc9-b6d2-7f7be344eb36 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.353427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4687-6f4e-4dc9-b6d2-7f7be344eb36 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.353427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3bc4687-6f4e-4dc9-b6d2-7f7be344eb36 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.355659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 468/950] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:02.361621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b07d-563f-44b0-9b80-2d14be3490fd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.362147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b07d-563f-44b0-9b80-2d14be3490fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.362147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c073b07d-563f-44b0-9b80-2d14be3490fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.382940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c073b07d-563f-44b0-9b80-2d14be3490fd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:02.384812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c073b07d-563f-44b0-9b80-2d14be3490fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:02.413608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:02.467593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 483/951] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:02.739470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc4687-6f4e-4dc9-b6d2-7f7be344eb36 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:02.857559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3bc4687-6f4e-4dc9-b6d2-7f7be344eb36 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:02.859108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:02.878412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aec3-5f4c-4b10-ac92-57568b6c6ec3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.878716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aec3-5f4c-4b10-ac92-57568b6c6ec3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.878960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e280aec3-5f4c-4b10-ac92-57568b6c6ec3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.881192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 484/952] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:02.886071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761260bb-a242-4b62-a6b6-1c1bf6b9a78c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.886652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761260bb-a242-4b62-a6b6-1c1bf6b9a78c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.886923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761260bb-a242-4b62-a6b6-1c1bf6b9a78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.895144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:02.895774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.896317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.896583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.897162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:02.898550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:02.906538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-761260bb-a242-4b62-a6b6-1c1bf6b9a78c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:02.907715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-761260bb-a242-4b62-a6b6-1c1bf6b9a78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:02.909005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:02.910165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:02.911132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 469/953] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:02 2022] GET /identity/v3/services/compute => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:02.915863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:02.982497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 485/954] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:02.983905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507886f-9bee-45e5-98dd-748779063498 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.984252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507886f-9bee-45e5-98dd-748779063498 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.984585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0507886f-9bee-45e5-98dd-748779063498 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:02.985895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 486/955] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:02.991062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:02.991282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:02.991552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.013850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:03.015111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:03.031544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd718f53-b5ca-4186-a284-df5e4d8d6578 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.068860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.069503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.069834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.070113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.070828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:03.071135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.072632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.079147 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 470/956] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:02 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:03.084445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.090815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 487/957] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:02 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:03.130129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.197416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8bc8c83-5668-49ea-b85b-158abc9ef79c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.233788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.234427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.234709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.234976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.235604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:03.235904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.237268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.242341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 471/958] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:03.277983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.305501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.391961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfec1e1-68ae-41a4-8911-be5cbb420f18 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.397207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.431329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.433105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.433629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.433906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.434571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.435986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:03.445519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:03.446789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:03.447636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 488/959] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:03.453545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.523068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.532693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e3f0-7538-4f0e-b9c9-31b8f300373f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.549243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.566193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0888ff48-fb49-4fd1-bb21-93479dddf29f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.577058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.577693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.577968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.578241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.591935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:03.599009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:03.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:03.600931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 472/960] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:03.607111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.610807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.611618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.612100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.612588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.613414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:03.613944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.615609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.626460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 489/961] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:03.628382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.721217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57dba89-156b-4830-81c7-893180d205c0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.747891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5ad1b6-b48d-4836-b62f-a2feeb50da8b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.759818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.760795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.761304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.761773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.762742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:03.763263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.764834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.774495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 473/962] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:03.780092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.790030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.792109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.792616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.793273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.794622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:03.795162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.797372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.801730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.810544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 490/963] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:03.814056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:03.832678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:03.915775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5761c191-2c49-4b9e-9783-c686b094623e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.926573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c790af9-8f9f-4a3d-80c0-c9cffc640837 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:03.955228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.956353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.956781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.957252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.964635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:03.965377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:03.965761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:03.966154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:03.967314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.968432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.971379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:03.973058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:03.990629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:03.991683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:03.992605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:03.992605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:03.992605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:03.992786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 491/964] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:07:03 2022] POST /identity/v3/endpoints => generated 349 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:07:04.002456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 474/965] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:04.003153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.087613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:04.131551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bb35e-3d03-4c4d-874f-deeb84f01fbe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:04.172298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:04.172938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.173220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.173490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.174162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:04.174542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:04.175931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2972d273-d934-4233-9094-ca0accb43dbd None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:04.190934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 492/966] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:04.196738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.271972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:04.279660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a25f-f0c8-457f-908b-68a785f7fde6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.279940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a25f-f0c8-457f-908b-68a785f7fde6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.280204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3077a25f-f0c8-457f-908b-68a785f7fde6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.284856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 493/967] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:04.287855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.288141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.288417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.309186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:04.310842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:04.313255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000bcd2c-41fd-4cff-a1c7-79a6f85518c6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:04.350629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:04.351613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.352105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.352568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.363568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:04.365296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:07:04.386323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:04.387378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Aug 30 04:07:04.388344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.391065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 475/968] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:04.404740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.412082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 494/969] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:04 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:04.518498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d9d6f0b-f44d-4072-90d2-2d9044a5f37e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:04.548097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:04.556094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:04.556711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.557547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.558183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.559295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:04.559607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:04.561036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:04.580865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 476/970] 10.222.0.46 () {56 vars in 1149 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:04.586279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.644185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:04.690295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9f9412-210c-4324-b84e-baccff3d0afb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:04.705900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.726813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:04.727421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.727696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.727964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.801325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:04.802819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:04.828065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0810351c-7bb3-4422-ae80-5476eaa2d536 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:04.861360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:04.869465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:04.870800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:04.871338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:04.871886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:04.872895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:04.874586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:04.878342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING keystone.server.flask.common [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Aug 30 04:07:04.893311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.sql.core [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 04:07:04.893311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 04:07:04.893311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [parameters: {'user_id': '55b9b1074aa24ba6932aae0c190474ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 04:07:04.893311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=81503) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 30 04:07:04.905043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return method(*args, **kwargs) Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:07:04.905963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application yield resource Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application next(self.gen) Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.rollback() Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.force_reraise() Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise self.value Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application session.commit() Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Aug 30 04:07:04.907010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._prepare_impl() Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.session.flush() Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._flush(objects) Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application compat.raise_( Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application flush_context.execute() Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rec.execute(self) Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Aug 30 04:07:04.907972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application _emit_insert_statements( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = connection._execute_20( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1705, in _execute_20 Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 333, in _execute_on_connection Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1572, in _execute_clauseelement Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ret = self._execute_context( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1943, in _execute_context Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2122, in _handle_dbapi_exception Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Aug 30 04:07:04.908996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1900, in _execute_context Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self.dialect.do_execute( Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result = self._query(query) Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application conn.query(q) Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application result.read() Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Aug 30 04:07:04.909885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application packet.raise_for_error() Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [parameters: {'user_id': '55b9b1074aa24ba6932aae0c190474ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:04.912558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Aug 30 04:07:04.914652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 477/971] 10.222.0.46 () {60 vars in 1131 bytes} [Tue Aug 30 04:07:04 2022] POST /identity/v3/users => generated 160 bytes in 202 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:04.917579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.928977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 495/972] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/auth/tokens => generated 1474 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:04.934722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:04.982507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.055534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d30b33-f6d1-4627-b6ed-249307bfa88f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.055761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b734788-e675-498f-880c-f3c6687c2ae8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.095038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.095778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.096125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.096464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.102834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.103697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.104311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.105244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.111086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.113189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.113734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.115424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:07:05.130837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:05.131861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Aug 30 04:07:05.132867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:05.135858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:05.137419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:05.138363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:05.138363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:05.138363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:05.138556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 478/973] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/users/cinder => generated 84 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:05.138556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.140825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 496/974] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:05.146151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.162415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.266750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17596ef-d472-4a04-a81e-286001ce9f9c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.277634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.291019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d2e033-20e6-4296-a1df-7465270e8d44 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.306236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.307016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.307576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.308096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.308823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:05.309343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.310721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f3e4f3-a1c4-4d9f-9f0d-d8f61bbbbc59 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.326918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.327591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.327935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.328372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.328972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:05.329335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.330766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b7c30-fc40-4efb-a8bf-cb0c7419b50f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.348459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 497/975] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:05.348459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8a83-75b0-4a77-819e-3f72f6fa6e8c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.348459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8a83-75b0-4a77-819e-3f72f6fa6e8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.348459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-899a8a83-75b0-4a77-819e-3f72f6fa6e8c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.348459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 498/976] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:05.351501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a8a83-75b0-4a77-819e-3f72f6fa6e8c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.357652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 479/977] 10.222.0.46 () {56 vars in 1101 bytes} [Tue Aug 30 04:07:05 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:05.362717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.362717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.362717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.387552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:05.389751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:05.436240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.478919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 480/978] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:05 2022] POST /identity/v3/auth/tokens => generated 1474 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:05.495410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-899a8a83-75b0-4a77-819e-3f72f6fa6e8c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.533980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.534659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.534936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.535202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.543069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.544529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:05.558056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:05.558994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:05.559902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:05.560946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 499/979] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:05.565547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.671938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.679046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7a423ea-ed3b-4d2b-9074-94055c1896d2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.715369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.715901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.716177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.716449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.717137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:05.717442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.718921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.735718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 481/980] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:07:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:05.737253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.756065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:05.849108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af7d29c-4b6e-404e-b7c3-784813069b53 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.870413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3cd0b68-dff7-4b85-9353-2f498fde487e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:05.877276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:05.886722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.887329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.887610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.887883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.888643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] RBAC: Adding query filter params (role.id=14e1f67917c94aee9b0c6c78e38790de, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=f3da3928b5414f14ae49e96282a135f1) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:05.888950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.890413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af85e456-dcac-4feb-9a42-4823c6f67c47 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.898427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 500/981] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:05 2022] GET /identity/v3/role_assignments?user.id=f3da3928b5414f14ae49e96282a135f1&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=14e1f67917c94aee9b0c6c78e38790de => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:05.921984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:05.922653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:05.922933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:05.923207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:05.923879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:05.925282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c1a63-c349-4fb8-a3df-9c5d46cc76fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:05.942283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 482/982] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:07:05 2022] POST /identity/v3/services => generated 242 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:06.082913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbae43-770a-4bbf-8a6e-bf66a5c3d306 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:06.083209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbae43-770a-4bbf-8a6e-bf66a5c3d306 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:06.083481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75bbae43-770a-4bbf-8a6e-bf66a5c3d306 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:06.084833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 501/983] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:06.090542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:06.090831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:06.091107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:06.111809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:06.113007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:06.127019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:06.206073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 483/984] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:06 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:06.473960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbae43-770a-4bbf-8a6e-bf66a5c3d306 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:06.594619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75bbae43-770a-4bbf-8a6e-bf66a5c3d306 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:06.628809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:06.629428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:06.629704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:06.629971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:06.630570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:06.631922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:06.641120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:06.642150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:06.647365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 502/985] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:07:06 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:06.648856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:06.749075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-246e5c7d-4e80-42ac-8dea-5f842f4d5b1c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:06.782668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:06.783265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:06.783538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:06.783802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:06.784417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:06.784714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:06.786061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:06.796103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 484/986] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:06 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:06.797515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:06.868013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:06.915571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad86b7d7-c461-4811-93c1-f49715f6c73d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:06.950587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:06.951194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:06.951467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:06.951731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:06.952355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:06.952652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:06.953999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:06.964051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 503/987] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:06 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:06.965040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:06.974119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:07.071360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f37713-b88a-4c8c-8ec0-e1017b8397de None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:07.107904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:07.108504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.108776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.109041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.109732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] RBAC: Adding query filter params (interface=admin, service_id=298d08872bc748ed85f41cf51776d6d8, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:07.110036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:07.111420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd90b337-36a5-4fd9-8f61-2f795b7dc2ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:07.121945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 485/988] 10.222.0.46 () {56 vars in 1249 bytes} [Tue Aug 30 04:07:06 2022] GET /identity/v3/endpoints?service_id=298d08872bc748ed85f41cf51776d6d8&interface=admin®ion_id=RegionOne => generated 193 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:07.123182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:07.261606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00dec51-95c0-4f88-b3b0-18fb6230e1ba None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:07.286119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:07.301099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:07.301708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.301985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.302279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.302872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:07.304220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b13a25b5-47f1-4853-95a0-c3112ca386e0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:07.318294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 504/989] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:07:07 2022] GET /identity/v3/services => generated 1727 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:07:07.936790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99fdd-ea6b-4438-b558-b8e8547d173f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.937336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99fdd-ea6b-4438-b558-b8e8547d173f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.937336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc99fdd-ea6b-4438-b558-b8e8547d173f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.938880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 486/990] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:07.944440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.944870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.945098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.950910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d6d9-8f51-490a-997d-8cade0c763f5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.951219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d6d9-8f51-490a-997d-8cade0c763f5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.951497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e26d6d9-8f51-490a-997d-8cade0c763f5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.952956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 487/991] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:07.958386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:07.958776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:07.959174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:07.967975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:07.970341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:07.980777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:07.982006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:08.044848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 505/992] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:08.046333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.054291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 488/993] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:07 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:08.324180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.325206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.427772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfad3945-d345-4d1e-ae98-2dc332b82b31 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.441333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27079d7b-a95a-41fa-8f6f-504691757551 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.443866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.466506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.467138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.467418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.467688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.468268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.469622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:08.478429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:08.480720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:08.481940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 506/994] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services/image => generated 86 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:08.484882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.485744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.485992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.486499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.487123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.487968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.489619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:08.499948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:08.500966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:08.503295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 489/995] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:08.506789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.574878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.602695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70939dc-2462-4b2c-b77a-59f493ceb1a7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.617899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8870a9-966d-4983-afb2-952adb3cac63 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.639532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.640702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.641074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.641563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.642302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:08.642722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.644163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.650289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 507/996] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:08.655771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.659790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.659958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.660220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.660551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.661241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] RBAC: Adding query filter params (name=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:08.661626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.663079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.673347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 490/997] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:08.674298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.704860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.768129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb808b9-2065-46f8-b2c9-e0d7b13bce34 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.784767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b52a61-0984-4bbf-8e72-d1cf1b417c15 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.802905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.803509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.803783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.804046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.804664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:08.804960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.806344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.826565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 508/998] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services?type=image => generated 122 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:08.827395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.832294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:08.833257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:08.833604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:08.833952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:08.834714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:08.835090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:08.836712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:08.846524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 491/999] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:08.849642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:08.961760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d506e-2f27-469e-b6d5-caadfaef3ada None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:08.970586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:08.999790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:09.000397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.000671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.000937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.001812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] RBAC: Adding query filter params (interface=public, service_id=af1ffc3913d0498ca1e8ee94c6561d7d, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:09.002118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:09.003499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cba43bd-bbbc-4160-8d4f-af99d329571b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:09.010573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 509/1000] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:07:08 2022] GET /identity/v3/endpoints?service_id=af1ffc3913d0498ca1e8ee94c6561d7d&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:09.015364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:09.123822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca15a9b-3967-497a-b50b-f1efd6b8179b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:09.161018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:09.161634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.161938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.162217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.162854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:09.164222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9ca7d4a-cffb-4778-be37-1191cd693899 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:09.172116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 492/1001] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:07:09 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:07:09.272848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f704-450c-4a95-94d8-5ea569b9ed49 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.273224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f704-450c-4a95-94d8-5ea569b9ed49 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.273402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b9f704-450c-4a95-94d8-5ea569b9ed49 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.274988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 510/1002] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:09.280558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.280894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.281116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.302833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:09.304408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:09.360708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:09.378832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 493/1003] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:09 2022] POST /identity/v3/auth/tokens => generated 1570 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:09.651359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f704-450c-4a95-94d8-5ea569b9ed49 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:09.755012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b9f704-450c-4a95-94d8-5ea569b9ed49 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:09.789592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:09.790215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.790517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.790784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.791896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:09.793392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:09.802518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:09.803578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Aug 30 04:07:09.804445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:09.807732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 511/1004] 10.222.0.46 () {56 vars in 1126 bytes} [Tue Aug 30 04:07:09 2022] GET /identity/v3/services/object-store => generated 93 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:09.814281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:09.917737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62d29a2-2ff1-43fd-b480-1cb07fa8b03a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:09.954664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:09.955275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:09.955551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:09.955822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:09.956449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:09.956751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:09.958124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:09.963043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 494/1005] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:09.969248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:10.003743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:10.072215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4457020-1bf2-4642-baf6-ffcc137ef232 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:10.107189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:10.107787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:10.108071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:10.108335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:10.108951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:10.109247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:10.110630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12465173-b560-4a21-aed3-ad61dd742d27 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:10.116554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 512/1006] 10.222.0.46 () {56 vars in 1122 bytes} [Tue Aug 30 04:07:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:10.121860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:10.177403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:10.231558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f2f30b-2714-4f2d-829c-66e67d94ea70 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:10.266982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:10.267601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:10.267882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:10.268154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:10.268836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:10.270206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7c7315-83c9-4697-bac7-01b179a6c89a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:10.294354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 495/1007] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:07:10 2022] POST /identity/v3/endpoints => generated 325 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:07:10.810344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77941e-5378-40ff-936a-b7d42bf319b2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:10.810344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77941e-5378-40ff-936a-b7d42bf319b2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:10.810344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea77941e-5378-40ff-936a-b7d42bf319b2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:10.813517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 513/1008] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:10.819040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd7344-9998-4bae-bcbc-086cb2187328 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:10.819320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd7344-9998-4bae-bcbc-086cb2187328 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:10.819591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35cd7344-9998-4bae-bcbc-086cb2187328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:10.839123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35cd7344-9998-4bae-bcbc-086cb2187328 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:10.840489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35cd7344-9998-4bae-bcbc-086cb2187328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:10.920309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 496/1009] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:10 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:11.152304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fbbbe-00f3-4142-8bc2-24fb3efb378a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:11.152826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fbbbe-00f3-4142-8bc2-24fb3efb378a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:11.152826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a36fbbbe-00f3-4142-8bc2-24fb3efb378a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:11.154378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 514/1010] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:11.159409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:11.159818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:11.159982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:11.178918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:11.180123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:11.191961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fbbbe-00f3-4142-8bc2-24fb3efb378a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:11.213958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:11.262472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 497/1011] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:11 2022] POST /identity/v3/auth/tokens => generated 1717 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:11.300353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:11.306123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a36fbbbe-00f3-4142-8bc2-24fb3efb378a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:11.342933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:11.343542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:11.343820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:11.344089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:11.344736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:11.346154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:11.370297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 515/1012] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:07:11 2022] POST /identity/v3/services => generated 242 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:07:11.532515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:11.634628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da97fbb5-58df-4da8-b878-38dc254fb28d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:11.680050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:11.680690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:11.680969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:11.681240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] PATH_INFO: `/v3/services/compute` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:11.681843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:11.683907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:11.693114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:11.694118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:11.699141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 498/1013] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:11.705853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:11.815081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a7838f-4594-4665-a25c-934d9ccbcf7e None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:11.852613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:11.853581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:11.854806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:11.855290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:11.856129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] RBAC: Adding query filter params (name=compute) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:11.856631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:11.858191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:11.868292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 516/1014] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:11.869928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:11.898886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:11.977085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706c1894-2bd0-4e42-b334-880433224748 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:12.013580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:12.014194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.014501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.014775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.015407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] RBAC: Adding query filter params (type=compute) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:12.015710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:12.017078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bbba45-57f8-44aa-a03b-69f1cbaf9d7c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:12.026985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 499/1015] 10.222.0.46 () {56 vars in 1112 bytes} [Tue Aug 30 04:07:11 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:12.028523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:12.054567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:12.135450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d826a44-24f5-4d29-9f57-571e6399e87b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:12.170977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:12.171600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.171849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.172161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.172828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:12.174167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed521a0f-3abe-457b-9182-fb19b1aa53a5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:12.198300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 517/1016] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:07:12 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:12.233609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbce49-d04a-4120-8b76-a1ef9ad97b76 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.234167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbce49-d04a-4120-8b76-a1ef9ad97b76 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.234167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cbce49-d04a-4120-8b76-a1ef9ad97b76 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.235735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 500/1017] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:12.241582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.241901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.242146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.262127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:12.263386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:12.352930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 518/1018] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:12 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:12.621701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbce49-d04a-4120-8b76-a1ef9ad97b76 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:12.716795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:12.729235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cbce49-d04a-4120-8b76-a1ef9ad97b76 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:12.767157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:12.767795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.768071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.768339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.769510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:12.771297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141a9e8b-1e2e-4929-810f-9ed839a4fa3b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:12.773083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 501/1019] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:12.779230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:12.782360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:12.892506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1454b3d5-24f2-4b45-92f3-50839602a765 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:12.926862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:12.927473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:12.927750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:12.928017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:12.928680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:12.930097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2621543f-a8b2-495e-8d54-40ac8b917bba None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:12.957628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 519/1020] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:07:12 2022] POST /identity/v3/projects => generated 311 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:13.310631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb0ca-1760-4b48-a909-7f728d69f425 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:13.311124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb0ca-1760-4b48-a909-7f728d69f425 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:13.311124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96eb0ca-1760-4b48-a909-7f728d69f425 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:13.312593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 502/1021] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:13.317534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:13.317933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:13.318099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:13.336045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:13.337229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:13.391730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:13.412175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 520/1022] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:13 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:13.679599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eb0ca-1760-4b48-a909-7f728d69f425 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:13.784298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96eb0ca-1760-4b48-a909-7f728d69f425 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:13.833577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:13.834433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:13.834715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:13.834981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] PATH_INFO: `/v3/services/image` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:13.835571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:13.836944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:13.847335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:13.850873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:13.851732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 503/1023] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 30 04:07:13 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:13.859263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:13.967278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee92dc-661e-4e20-b1e9-784cf8b667aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.004780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.005405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.005681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.005948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.007351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] RBAC: Adding query filter params (name=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:14.007351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:14.008307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:14.019149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 521/1024] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:14.020491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:14.050869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:14.146806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5477f5-864b-41c9-80cf-6cfd79530cf9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.186653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57864769-8af5-4638-994e-dee8354382ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.187602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864769-8af5-4638-994e-dee8354382ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.188054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864769-8af5-4638-994e-dee8354382ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.188498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57864769-8af5-4638-994e-dee8354382ef None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.189299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864769-8af5-4638-994e-dee8354382ef None admin] RBAC: Adding query filter params (type=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:14.189784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864769-8af5-4638-994e-dee8354382ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:14.191343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57864769-8af5-4638-994e-dee8354382ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:14.196767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 504/1025] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/services?type=image => generated 350 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:14.203108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:14.263638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:14.268089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6233e-549d-49a5-89fa-772f94deede1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.268373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6233e-549d-49a5-89fa-772f94deede1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.268650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b6233e-549d-49a5-89fa-772f94deede1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.270001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 505/1026] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:14.275615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.276401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.276702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.295731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:14.296938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:14.329971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f059919f-3ecb-4f67-a576-7ed0f37bdf8f None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.365338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.365945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.366217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.366506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.367198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] RBAC: Adding query filter params (interface=public, service_id=ab486517b7014baabccc7d48a9631c05, region_id=RegionOne) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:14.367581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:14.368857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:14.373963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 506/1027] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:14.374576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 522/1028] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/endpoints?service_id=ab486517b7014baabccc7d48a9631c05&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:14.383704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:14.403644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:14.487834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36dec2a5-beab-4ed0-8c0d-59cb38995449 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.522561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.523085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.523358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.523625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.524225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:14.526188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:14.538525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 507/1029] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:07:14.645426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:14.754267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3172ceb1-8a63-4fb5-86e2-09e16933afb2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.769151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:14.792020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.792574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.792904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.793183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.803686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-625b56f3-9f31-4618-a7b3-7b56b261f59d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:14.812322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:14.813303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:14.814278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 523/1030] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:14.817881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:14.911579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:14.933948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23af9c4-2a15-4d3c-b689-91eaee842247 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:14.949553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb47f-7cb2-402d-a71e-dff4d8ad8d64 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.949866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb47f-7cb2-402d-a71e-dff4d8ad8d64 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.950422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fbb47f-7cb2-402d-a71e-dff4d8ad8d64 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.952145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 524/1031] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:14.957236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.957515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.957782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.970958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:14.971573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:14.971849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:14.972117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:14.972785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:14.973087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:14.974486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:14.976590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:14.977826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:14.984100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 508/1032] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:14.989658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.053985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 525/1033] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:14 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:15.107916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b97f429-f311-4e31-adea-270e1888494c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.143199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.143838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.144118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.144385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.156000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.157454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.159298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:15.174592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:15.175779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:15.176728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:15.176728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:15.176728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:15.176728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 509/1034] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:15.181812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.286728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e72313-d18c-4c62-8d87-5b761830f44e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.289003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:15.322933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.323533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.323814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.324086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.324730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:15.325033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.326522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.326631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.344004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 526/1035] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:15.344868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.436386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab8bf4f-6dad-48a3-88c4-1ec542db49f0 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.453028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d61062d-7cbb-4784-944a-45d0f1216ecb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.474046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.474912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.475329 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.475864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.477225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.478994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.485387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 510/1036] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:15.486212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.487296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.487934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.488209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.488474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.496702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.498132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:07:15.509450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:15.516172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:15.517268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 527/1037] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:07:15 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:15.519666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.546981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:15.563199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:15.609774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97adcd-da4c-469f-8ac1-117df33a7182 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.637442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed04dd7-ddf9-4eaa-9504-711b72082998 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.651625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.652341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.652616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.652875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.653519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.655286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.673872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.674497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.674777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.675044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.675768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:15.676072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.677543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:15.686990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 511/1038] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:07:15 2022] POST /identity/v3/users => generated 285 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:15.698817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 528/1039] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:07:15 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:15.705087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:15.822281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98433d42-56e6-4ff0-a7b4-bcdf7e74b0dd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:15.840911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:15.864880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:15.865422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:15.865696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:15.865964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:15.938481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:15.939988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.040773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 512/1040] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:15 2022] GET /identity/v3/auth/tokens => generated 1969 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 04:07:16.046963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.130107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:16.151684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046f0fa1-a937-47fc-b90e-5df7650bb269 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:16.185788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:16.186444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.186738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.186981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.197215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:16.199613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:16.215924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:16.216955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:16.217878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:16.217878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:16.217878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:16.217878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 529/1041] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:16.224186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.243967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:16.330962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024a3bb5-d74d-4270-b519-55a7d1e5d5cf None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:16.368495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:16.369181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.369499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.369767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.370440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:16.370739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:16.372312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ef42871-4233-48f2-8cdf-571905d40aa5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.389372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 513/1042] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:16.390946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.506766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0aeed0-0e86-4aa0-84d6-5e9595927e31 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:16.526388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee609dc-b41e-48e8-b1d5-869e6e52bd8b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.527146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee609dc-b41e-48e8-b1d5-869e6e52bd8b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.527146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee609dc-b41e-48e8-b1d5-869e6e52bd8b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.528437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 514/1043] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:16.536407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.536702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.536975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.548242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:16.548877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.549145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.549407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.551550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:16.558686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:16.560152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:16.561803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:16.564430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:16.577689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:16.578800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:16.579720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 530/1044] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:16.584641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.644006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 515/1045] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:16 2022] POST /identity/v3/auth/tokens => generated 1969 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:16.721187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6debb0-6c5b-4837-9d80-7ec1af839897 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:16.758862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:16.759473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.759754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.760174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.761090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:16.761629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:16.764336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.783624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 531/1046] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:16.789831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.802064 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:16.911711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-272e9588-a41a-48db-83ce-a4b7e3c2dca4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:16.953608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:16.954266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:16.954573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:16.954846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:16.955879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] RBAC: Adding query filter params (role.id=14e1f67917c94aee9b0c6c78e38790de, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:16.956192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:16.957659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:16.958524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:16.968443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 516/1047] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=14e1f67917c94aee9b0c6c78e38790de => generated 259 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:17.032808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:17.086661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00bff02-9469-45be-a293-9af7628d3c19 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:17.127450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:17.127660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.127880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.128099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] PATH_INFO: `/v3/services/image` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:17.128685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:17.130054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:17.142830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:17.143927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Aug 30 04:07:17.144782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:17.145539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 532/1048] 10.222.0.46 () {56 vars in 1105 bytes} [Tue Aug 30 04:07:16 2022] GET /identity/v3/services/image => generated 86 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:17.152565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:17.238975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:17.275225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c66294-9518-4a8e-905d-9be8ed0908b4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:17.313312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:17.313921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.314196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.314489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:17.315123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] RBAC: Adding query filter params (name=image) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:17.315422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:17.316778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:17.327269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 517/1049] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:17 2022] GET /identity/v3/services?name=image => generated 122 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:17.328223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:17.430165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36edf0aa-d667-4198-9825-6cde8ac31462 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:17.450059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:17.468242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:17.468846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.469124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.469398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:17.470014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] RBAC: Adding query filter params (type=image) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:17.470334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:17.471697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2f0f5b-2b51-4899-af9c-2895508a3ce2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:17.476952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 533/1050] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:17 2022] GET /identity/v3/services?type=image => generated 350 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:17.483925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:17.587625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2bdca15-f3ac-45e1-8943-84a9f1c9273e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:17.644908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:17.645531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.646133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.646508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:17.647218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:17.648586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9ccee7-4f64-49e1-847a-0b3989fd8090 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:17.654437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:17.675943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 518/1051] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:07:17 2022] POST /identity/v3/endpoints => generated 327 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:17.979536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e7f42-371f-4e97-953b-c29a1f49bd90 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.979850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e7f42-371f-4e97-953b-c29a1f49bd90 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.980034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6e7f42-371f-4e97-953b-c29a1f49bd90 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:17.981410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 534/1052] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:17.987000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:17.987360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:17.987705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.007178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:18.008368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:18.090305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 519/1053] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:18.354043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e7f42-371f-4e97-953b-c29a1f49bd90 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:18.462301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6e7f42-371f-4e97-953b-c29a1f49bd90 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:18.498479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:18.499112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:18.499391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:18.499661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.510699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:18.515592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:18.516506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:18.517358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:18.522150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 535/1054] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:18.523172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:18.632383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5130e98-878f-44f5-b259-310c57b2bf13 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:18.652594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:18.671597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:18.672586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:18.672868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:18.673136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.673805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:18.674107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:18.675508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:18.689196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 520/1055] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:18.690905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:18.737826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:18.799673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cd650-93f1-4afa-9483-75b4f35f2a8a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:18.835723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:18.836359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:18.836636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:18.836910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] PATH_INFO: `/v3/users/9550c8dc66e447e6ad902ff8efb44789` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.851271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] RBAC: Authorizing `identity:get_user(user_id=9550c8dc66e447e6ad902ff8efb44789)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:18.854837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c4520f-104f-4bc4-80c7-01e440d2e689 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:18.861539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 536/1056] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:18 2022] GET /identity/v3/users/9550c8dc66e447e6ad902ff8efb44789 => generated 285 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:18.862965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:18.921997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b66c0-9b75-455c-ab84-a11decce03cd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:18.922402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b66c0-9b75-455c-ab84-a11decce03cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:18.922824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5b66c0-9b75-455c-ab84-a11decce03cd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.924443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 537/1057] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:18.929543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:18.929924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:18.930266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:18.949938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:18.951177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:18.976821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd5edb8-94a7-44c7-8c2b-a55fabc2bdbe None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.015592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.016205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.016596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.016872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.029811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 538/1058] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:18 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:19.072585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.094960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:19.096409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:19.202229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 521/1059] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:18 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:19.208077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.240429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.300837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.317466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f2d350-1ccf-48e0-8496-fab14e50ccab None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.352313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.352940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.353218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.353487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.355290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:19.356718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:19.358689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 539/1060] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:19.364184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.391104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.413839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a633370d-b254-4f48-9226-bc93ae1cc8b1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.452950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.453517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.454170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.454502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.465898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:19.469775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:19.470756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:19.471709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:19.472579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 522/1061] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:19.477940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.480051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d05ed7-9039-4376-a737-befd66387fb1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.520570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.521438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.521715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.521983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.522730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=3e581b76eb774bafafaec25e5e5d2301, user.id=9550c8dc66e447e6ad902ff8efb44789) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:19.523035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:19.524444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d96c82-dcb5-460b-a7d5-21f00513025f None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:19.531297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 540/1062] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/role_assignments?user.id=9550c8dc66e447e6ad902ff8efb44789&scope.project.id=3e581b76eb774bafafaec25e5e5d2301&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:19.597430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fedfaf-abd7-4910-8e4f-42bb26abeabd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.636603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.637248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.637541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.637751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.638453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:19.638756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:19.640159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:19.644464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da137-7044-4fd3-9671-2153a792906e None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.644743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da137-7044-4fd3-9671-2153a792906e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.645026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d9da137-7044-4fd3-9671-2153a792906e None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.646789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 541/1063] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:19.651649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 523/1064] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:19.653396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.653681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.653955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.659260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.663199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.675916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:19.677697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:19.759465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 542/1065] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:19 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:19.780602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a05c3d-b04b-4ed7-b1f2-26c822e18a40 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:19.824093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:19.824738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:19.825019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:19.825294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:19.836490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:19.838506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:19.854297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:19.861134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:19.862122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:19.862122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:19.862122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:19.862122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 524/1066] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:19.862678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:19.958381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:19.971685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d732a51-7bec-47fc-9c36-7a78483c4956 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.007094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.007705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.007983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.008252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.008893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:20.009191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.011042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.028558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 543/1067] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:20.029702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.032057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.099685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.147801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368b53d9-d516-48d9-92fe-237305ac2c40 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.152144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea28629-9be9-4bd6-8f2c-60bdc59a5d89 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.183578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.184181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.184459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.184726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.189639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.190285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.190569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.190838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.199866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.201312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.207190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:07:20.213272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:20.214549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:20.215504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 525/1068] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:20.220199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.271838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.273210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.332077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16bcc909-96d1-489e-9c75-e6ee66f2d3f9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.346432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.368841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.370039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.370576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.370924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.371668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:20.371975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.373420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.378296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 544/1069] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 04:07:20.385242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.393581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 526/1070] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:20.399586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.446876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.505836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d03c812-aee2-45dd-8638-fa78571ecd93 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.513287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45f25fd-d43f-46f7-a4eb-7fb12df0797d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.544057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.544695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.544972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.545239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.549951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.550857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.551337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.551826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.556053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.558052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.563345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.564969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:20.574977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:20.575965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:20.576891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:20.576891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:20.576891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:20.576891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 527/1071] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:20.581823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:20.583013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:20.583940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:20.590813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:20.590813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:20.590813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:20.590813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 545/1072] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:20.590813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.608676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.696158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831ce203-3be3-455c-9b7d-0fdb8c325184 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.697886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4e33ea-4552-465b-9704-159f25fbdf64 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.700552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.735413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.736019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.736295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.736562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.736908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a056d-a018-470c-9518-3f776cece830 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.737206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:20.737510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.737616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a056d-a018-470c-9518-3f776cece830 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.737780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a056d-a018-470c-9518-3f776cece830 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.738050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054a056d-a018-470c-9518-3f776cece830 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.738730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a056d-a018-470c-9518-3f776cece830 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:20.738908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.739021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a056d-a018-470c-9518-3f776cece830 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.740363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054a056d-a018-470c-9518-3f776cece830 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.754285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 528/1073] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:20.754455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 546/1074] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:20.763733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:20.864456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:20.885588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee76fc-2575-4cbb-915a-6e98fe40b492 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:20.925813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:20.926581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:20.926864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:20.927134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:20.934826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:20.936283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:20.949202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:20.950177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:20.951299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 529/1075] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:20.958154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a056d-a018-470c-9518-3f776cece830 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:21.071940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054a056d-a018-470c-9518-3f776cece830 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:21.106184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:21.106817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:21.107090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:21.107352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:21.108036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:21.108335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:21.109733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceed2b3a-c1c3-44f8-a732-957198797701 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:21.122412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 547/1076] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:07:20 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:21.128558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:21.229542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ae4469-ccf9-4189-8eec-154e810654cc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:21.262844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:21.263497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:21.263774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:21.264046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/14e1f67917c94aee9b0c6c78e38790de` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:21.266176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=148e575ef9b642c8b81d7eca8dd70bfa, role_id=14e1f67917c94aee9b0c6c78e38790de)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:21.267633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5e57cf-2ccb-48a0-940e-bf010f7db8ff None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:21.281601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 530/1077] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:07:21 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/148e575ef9b642c8b81d7eca8dd70bfa/roles/14e1f67917c94aee9b0c6c78e38790de => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:07:21.505234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e565-cf9d-4b0c-b784-78f5f7f3d3dd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:21.505520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e565-cf9d-4b0c-b784-78f5f7f3d3dd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:21.505796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c7e565-cf9d-4b0c-b784-78f5f7f3d3dd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:21.507196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 548/1078] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:21.513343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:21.513622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:21.513917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:21.522036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:21.533848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:21.535075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:21.603364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 531/1079] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:21.874614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7e565-cf9d-4b0c-b784-78f5f7f3d3dd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:21.901477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:21.977973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c7e565-cf9d-4b0c-b784-78f5f7f3d3dd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:22.010888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:22.011503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:22.011776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:22.012040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:22.022301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:22.027686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:22.028854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:22.029890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 549/1080] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:22.034814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:22.143591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42778883-2dda-4c59-a641-397f0f90e3f5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:22.177808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:22.178438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:22.178713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:22.178980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:22.179631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:22.179928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:22.181282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:22.182664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:22.190101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 532/1081] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:22.195906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:22.280807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:22.297522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37281382-c29e-415f-b197-35acd3eb9929 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:22.331028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:22.331653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:22.331930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:22.332201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] PATH_INFO: `/v3/users/9550c8dc66e447e6ad902ff8efb44789` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:22.333310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] RBAC: Authorizing `identity:get_user(user_id=9550c8dc66e447e6ad902ff8efb44789)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:22.334762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:22.336346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 550/1082] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:22 2022] GET /identity/v3/users/9550c8dc66e447e6ad902ff8efb44789 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:22.342600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:22.453133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395e1356-5b28-4e2d-bff8-d5c1300880d3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:22.487597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:22.488221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:22.488501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:22.488774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:22.489884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:22.491345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d5d325-62f6-4b67-88c7-58afb4b2df00 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:22.492975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 533/1083] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:22 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:22.499219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:22.599661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332ec7b0-5bd4-4585-b38f-c4dce85629ad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:22.632820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:22.633471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:22.633751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:22.634024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301/users/9550c8dc66e447e6ad902ff8efb44789/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:22.636524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] RBAC: Authorizing `identity:create_grant(project_id=3e581b76eb774bafafaec25e5e5d2301, user_id=9550c8dc66e447e6ad902ff8efb44789, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:22.637946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f9fe77d-f86b-4286-afde-be1a89733fad None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:22.652051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 551/1084] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:07:22 2022] PUT /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301/users/9550c8dc66e447e6ad902ff8efb44789/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:07:23.236749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0ac0-4c7a-44f1-b72f-62292e939ab9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:23.237285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0ac0-4c7a-44f1-b72f-62292e939ab9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:23.237285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e55e0ac0-4c7a-44f1-b72f-62292e939ab9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:23.238647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 534/1085] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:23.244357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:23.245015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:23.245015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:23.263378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:23.264553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:23.271934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:23.331412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 552/1086] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:23 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:23.610958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e0ac0-4c7a-44f1-b72f-62292e939ab9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:23.672914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:23.714416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e55e0ac0-4c7a-44f1-b72f-62292e939ab9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:23.748504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:23.749129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:23.749405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:23.749750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:23.759799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:23.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:23.765677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:23.766561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 535/1087] 10.222.0.46 () {56 vars in 1120 bytes} [Tue Aug 30 04:07:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:23.772285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:23.882191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aaa7202-8c13-4390-93de-9b888e2fa7f0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:23.917747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:23.918378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:23.918657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:23.918923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:23.919580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:23.919897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:23.921230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:23.929158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 553/1088] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:07:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:23.935829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.040078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3d7b0-0d5f-47f2-a576-024d1e45271f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:24.070282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:24.082695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:24.083321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.083597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.083866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.094037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:24.095513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:24.110409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:24.116499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:24.117673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:24.117673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:24.117673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:24.117673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 536/1089] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:24.117929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.121749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:24.228357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33807c78-baaa-4fe0-829d-8802a982adaf None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:24.261366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:24.262003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.262234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.262533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.263171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:24.263470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:24.264819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:24.279640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 554/1090] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:24.285123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.385905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab777c8a-b639-4e4f-abf4-c49025285b8a None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:24.419095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:24.419721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.419998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.420269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.427694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:24.429127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:07:24.440729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:24.441884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:24.442947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 537/1091] 10.222.0.46 () {56 vars in 1093 bytes} [Tue Aug 30 04:07:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:24.448124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.509676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:24.548585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402ff5d8-289c-42da-bf4b-08de2e9450fe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:24.581812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:24.582438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.582716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.582983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.583697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:24.583995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:24.585398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f881603-cda2-4d25-a6d9-9d03680e17d1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:24.604200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 555/1092] 10.222.0.46 () {56 vars in 1133 bytes} [Tue Aug 30 04:07:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:24.609748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.643951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:24.662324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0a970-ca1d-4e4f-a5e9-15390c55630b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.662610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0a970-ca1d-4e4f-a5e9-15390c55630b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.662904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f0a970-ca1d-4e4f-a5e9-15390c55630b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.664237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 556/1093] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:07:24.669873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.670152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.670465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.688242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:24.689428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:24.710608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b49ca3ed-3e72-421b-a2f8-b0b233ac82f4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:24.745657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:24.746277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:24.746662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:24.746943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:24.759415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 557/1094] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:24 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:07:24.820088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:24.821816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:24.855595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:24.924975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 538/1095] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:24 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:24.931465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:24.982955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.028336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.029521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8248b70-bbff-4b55-8996-bfc85da1135b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.067157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.067703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.067980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.068249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.080200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.081646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:25.096551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:25.097528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:07:25.098493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:25.098493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:07:25.098493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:25.098493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 558/1096] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:25.103387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.132526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ca7610-8011-483c-98dd-eb05be1920e9 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.167628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.168257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.168537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.168840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.171500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.180358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:25.185814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:25.186955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:25.188326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 539/1097] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:25.196350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.211329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c988f4-16fc-4d7a-a2bb-07e94738019a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.214507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.251864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.252473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.252752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.253023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.253660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:25.253962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.255366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.267901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 559/1098] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:25.273435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.306848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb661b48-cf66-424d-9791-32e2db80f398 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.341175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.341781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.342068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.342362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.343027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:25.343331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.344698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.353212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 540/1099] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:25.360599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.375050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9ecad2-222d-4411-b67f-0e40b105d1eb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.411949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.412620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.412960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.413227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.420820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.421275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.422409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:07:25.434498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:25.435582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:25.436519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 560/1100] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:07:25.441763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.466072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182f3681-a6f6-4b59-9498-7d6f1d730e23 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.467562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.501414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.502045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.502370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.502635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] PATH_INFO: `/v3/users/9550c8dc66e447e6ad902ff8efb44789` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.503765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] RBAC: Authorizing `identity:get_user(user_id=9550c8dc66e447e6ad902ff8efb44789)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.505190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.506808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 541/1101] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/users/9550c8dc66e447e6ad902ff8efb44789 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:25.512937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.547829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dde37b4-b801-4ee7-b63b-99419f255a5b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.582199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.582821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.583100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.583368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.584054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:25.584356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.585775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.598199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 561/1102] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:25.604778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.620985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86418244-ccde-4907-9ef4-f982da87dde8 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.645630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.655446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.656049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.656331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.656603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.688164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:25.704530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095eb5ef-b7b0-4248-8714-a4416ba24fbe None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.724910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.726369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.739739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.740342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.740619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.740888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.741615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] RBAC: Adding query filter params (role.id=14e1f67917c94aee9b0c6c78e38790de, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=148e575ef9b642c8b81d7eca8dd70bfa) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:25.741925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.743484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.750166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 562/1103] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/role_assignments?user.id=148e575ef9b642c8b81d7eca8dd70bfa&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=14e1f67917c94aee9b0c6c78e38790de => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:07:25.835865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 542/1104] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Aug 30 04:07:25.842203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:25.948773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a3bc6a1-4f6c-4025-aa16-577ee136e6e0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:25.983435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:25.984071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:25.984357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:25.984631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:25.985755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:25.987238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a97bfd-7c31-4010-aea9-2411964727ef None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:25.988873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 563/1105] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:25.995990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:26.024350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:26.100425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834bf161-9f07-4393-9691-b93d2778bbd2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:26.135502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:26.136404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:26.136679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:26.136952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:26.137694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=3e581b76eb774bafafaec25e5e5d2301, user.id=9550c8dc66e447e6ad902ff8efb44789) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:26.138008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:26.139717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31bd0147-100f-42f8-9c76-3525285c6573 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:26.146804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 543/1106] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:25 2022] GET /identity/v3/role_assignments?user.id=9550c8dc66e447e6ad902ff8efb44789&scope.project.id=3e581b76eb774bafafaec25e5e5d2301&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:28.096793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827393-43b7-4b18-9c0f-c51cec109a70 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:28.097312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827393-43b7-4b18-9c0f-c51cec109a70 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:28.097429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12827393-43b7-4b18-9c0f-c51cec109a70 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:28.098703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 564/1107] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:28.104482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:28.104907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:28.105101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:28.122691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:28.123857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:28.188706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 544/1108] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:28 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:28.457289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12827393-43b7-4b18-9c0f-c51cec109a70 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:28.475469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:28.560113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12827393-43b7-4b18-9c0f-c51cec109a70 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:28.595966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:28.596582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:28.596853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:28.597150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:28.598284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:28.600479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1aaf50b-9f32-49cc-8c0c-da03d35af262 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:28.602533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 565/1109] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:28.608961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:28.630611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:28.707990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02434b0f-4359-4f90-9286-ec87cdf4343e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:28.740945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:28.741543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:28.741820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:28.742093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:28.742758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:28.746093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e04a638-4c28-42d4-8031-205c5c883cdb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:28.766032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 545/1110] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:07:28 2022] POST /identity/v3/users => generated 286 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:30.770167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece627-8328-4331-b5af-04f6c0c63923 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:30.771099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece627-8328-4331-b5af-04f6c0c63923 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:30.771099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbece627-8328-4331-b5af-04f6c0c63923 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:30.772332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 566/1111] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:30.777639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:30.778079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:30.778181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:30.796369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:30.797538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:30.863091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 546/1112] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:30 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:31.138484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbece627-8328-4331-b5af-04f6c0c63923 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:31.226928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:31.244068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbece627-8328-4331-b5af-04f6c0c63923 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:31.276097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:31.276873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:31.277146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:31.277410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] PATH_INFO: `/v3/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:31.278804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] RBAC: Authorizing `identity:get_role(role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:31.280154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:31.281449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 567/1113] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:31 2022] GET /identity/v3/roles/793acbb060a2436192eb2c7413739e83 => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:31.287224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:31.296176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:31.385938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961adf91-5d59-486f-940f-f8fe9c8bfae8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:31.420197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:31.420753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:31.421033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:31.421307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] PATH_INFO: `/v3/users/595e6a765c814173aaa9a17a507d9d5a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:31.434989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] RBAC: Authorizing `identity:get_user(user_id=595e6a765c814173aaa9a17a507d9d5a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:31.436423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:31.438031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 547/1114] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:31 2022] GET /identity/v3/users/595e6a765c814173aaa9a17a507d9d5a => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:31.443879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:31.542646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a47429b-4c40-4669-a393-15934946aae3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:31.574920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287480-b087-4947-866a-87c124078c27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:31.575509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287480-b087-4947-866a-87c124078c27 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:31.575794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287480-b087-4947-866a-87c124078c27 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:31.576061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87287480-b087-4947-866a-87c124078c27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:31.596815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:31.642016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87287480-b087-4947-866a-87c124078c27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:31.643488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87287480-b087-4947-866a-87c124078c27 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:31.738917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 568/1115] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:31 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 295 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:31.744789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:31.805531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:31.844359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0fd517-a797-40f0-90ce-6abbdbe50923 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:31.876998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:31.877612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:31.877886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:31.878152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:31.879280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:31.880723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192c3e3b-5363-47ef-8612-3154e7e84def None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:31.882356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 548/1116] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:31 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:31.889359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287480-b087-4947-866a-87c124078c27 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:31.941443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:31.988597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87287480-b087-4947-866a-87c124078c27 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:32.022613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:32.023140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:32.023416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:32.023681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:32.024388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=3e581b76eb774bafafaec25e5e5d2301, user.id=595e6a765c814173aaa9a17a507d9d5a) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:32.024691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:32.026294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88329c32-2e9c-4a58-83bf-945e7ad8ddb0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:32.031476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 569/1117] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:31 2022] GET /identity/v3/role_assignments?user.id=595e6a765c814173aaa9a17a507d9d5a&scope.project.id=3e581b76eb774bafafaec25e5e5d2301&role.id=793acbb060a2436192eb2c7413739e83 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:33.906729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b0330-b2cb-493b-a9f4-763e8fa08ebe None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:33.907239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b0330-b2cb-493b-a9f4-763e8fa08ebe None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:33.907353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598b0330-b2cb-493b-a9f4-763e8fa08ebe None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:33.908616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 549/1118] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:33.913742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:33.914182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:33.914347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:33.932426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:33.933610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:33.998336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 570/1119] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:33 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:34.266763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b0330-b2cb-493b-a9f4-763e8fa08ebe None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:34.370490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598b0330-b2cb-493b-a9f4-763e8fa08ebe None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:34.399323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:34.405127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:34.405741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:34.406016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:34.406305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] PATH_INFO: `/v3/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:34.407693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] RBAC: Authorizing `identity:get_role(role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:34.409045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:34.410549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 550/1120] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:34 2022] GET /identity/v3/roles/793acbb060a2436192eb2c7413739e83 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:34.416771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:34.462219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:34.514609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1b725b-d276-4afc-b935-a6f739119084 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:34.549589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:34.550149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:34.550456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:34.550726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] PATH_INFO: `/v3/users/595e6a765c814173aaa9a17a507d9d5a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:34.551849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] RBAC: Authorizing `identity:get_user(user_id=595e6a765c814173aaa9a17a507d9d5a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:34.553260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:34.554870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 571/1121] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:34 2022] GET /identity/v3/users/595e6a765c814173aaa9a17a507d9d5a => generated 286 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:34.561695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:34.660933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d52a8b6-cc9a-404d-81e9-c0462557bfef None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:34.694591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:34.695205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:34.695483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:34.695754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:34.696870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:34.698660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b98284-3ff7-46f7-b655-3e6299504c19 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:34.700286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 551/1122] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:34 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:34.706193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:34.803109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0759e6-7666-4ed3-9434-ccd46c72fbc0 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:34.818779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:34.839651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:34.840297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:34.840574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:34.840842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301/users/595e6a765c814173aaa9a17a507d9d5a/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:34.842980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] RBAC: Authorizing `identity:create_grant(project_id=3e581b76eb774bafafaec25e5e5d2301, user_id=595e6a765c814173aaa9a17a507d9d5a, role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:34.844394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6bc739f-90be-4631-a06b-0a3cc2e68ab2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:34.857599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 572/1123] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:07:34 2022] PUT /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301/users/595e6a765c814173aaa9a17a507d9d5a/roles/793acbb060a2436192eb2c7413739e83 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:07:36.769062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644153a6-b806-48c4-a147-82a6f4e862be None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:36.769614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644153a6-b806-48c4-a147-82a6f4e862be None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:36.769614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644153a6-b806-48c4-a147-82a6f4e862be None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:36.771240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 552/1124] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:36.776644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:36.777085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:36.777189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:36.795815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:36.796996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:36.864111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 573/1125] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:36 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:37.136902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644153a6-b806-48c4-a147-82a6f4e862be None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:37.206854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:37.240744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644153a6-b806-48c4-a147-82a6f4e862be None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:37.274270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:37.274899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:37.275701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:37.275969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] PATH_INFO: `/v3/roles/793acbb060a2436192eb2c7413739e83` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:37.277348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] RBAC: Authorizing `identity:get_role(role_id=793acbb060a2436192eb2c7413739e83)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:37.278732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:37.280046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 553/1126] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:37 2022] GET /identity/v3/roles/793acbb060a2436192eb2c7413739e83 => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:37.285884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:37.394111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d17fcda-6faa-4d95-abda-774a13344123 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:37.396557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:37.430071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:37.430727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:37.431006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:37.431328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] PATH_INFO: `/v3/users/595e6a765c814173aaa9a17a507d9d5a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:37.432606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] RBAC: Authorizing `identity:get_user(user_id=595e6a765c814173aaa9a17a507d9d5a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:37.434016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:37.435622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 574/1127] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:37 2022] GET /identity/v3/users/595e6a765c814173aaa9a17a507d9d5a => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:37.441391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:37.540163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb6357d-78a6-45ae-bdea-eaf59bce6b82 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:37.573083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:37.573674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:37.573947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:37.574218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:37.577633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:37.641779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:37.643267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:37.748239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 554/1128] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:37 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:37.754089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:37.855225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f97907e-f7ec-46df-a3f7-84c834da9100 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:37.887487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:37.888101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:37.888376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:37.888642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] PATH_INFO: `/v3/projects/3e581b76eb774bafafaec25e5e5d2301` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:37.889736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] RBAC: Authorizing `identity:get_project(project_id=3e581b76eb774bafafaec25e5e5d2301)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:37.891178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5af7613-4b61-40b9-aea3-873348bc2582 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:37.892763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 575/1129] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:37 2022] GET /identity/v3/projects/3e581b76eb774bafafaec25e5e5d2301 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:37.899240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:37.933382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:38.003220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94035b18-246b-4af3-96a3-bbc5488a2ecb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:38.036912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:38.037502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:38.037776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:38.038040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:38.038784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] RBAC: Adding query filter params (role.id=793acbb060a2436192eb2c7413739e83, scope.project.id=3e581b76eb774bafafaec25e5e5d2301, user.id=595e6a765c814173aaa9a17a507d9d5a) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:38.039085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:38.040491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d1ea08-4c3d-4c89-bcb9-bbc5babc8242 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:38.047021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 555/1130] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:37 2022] GET /identity/v3/role_assignments?user.id=595e6a765c814173aaa9a17a507d9d5a&scope.project.id=3e581b76eb774bafafaec25e5e5d2301&role.id=793acbb060a2436192eb2c7413739e83 => generated 608 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:39.999659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334db8-08c6-4a70-994d-fe2c0b64e0e2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:40.000223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334db8-08c6-4a70-994d-fe2c0b64e0e2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:40.000223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5334db8-08c6-4a70-994d-fe2c0b64e0e2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:40.001545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 576/1131] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:40.006866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:40.007285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:40.007485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:40.028410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:40.029661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:40.097210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 556/1132] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:40 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:40.365506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334db8-08c6-4a70-994d-fe2c0b64e0e2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:40.408495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:40.473159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5334db8-08c6-4a70-994d-fe2c0b64e0e2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:40.505131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:40.505738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:40.506010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:40.506293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:40.507380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:40.508782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb08e9c5-dbff-4cde-bf91-d3456dd215c7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:40.510878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 577/1133] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:40.516814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:40.546218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:40.614343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a50e0834-0664-4424-9a1b-ffe940d363b0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:40.652484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:40.653078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:40.653355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:40.653626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:40.654349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:40.655709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c96b86b5-bb9d-4034-9254-b91c4af61eac None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:40.683031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 557/1134] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:07:40 2022] POST /identity/v3/projects => generated 311 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:07:42.622856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb3d4-0a67-4478-927d-95cbdb9e3907 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:42.623413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb3d4-0a67-4478-927d-95cbdb9e3907 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:42.623413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a3eb3d4-0a67-4478-927d-95cbdb9e3907 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:42.624717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 578/1135] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:42.629771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:42.630210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:42.630341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:42.648758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:42.650473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:42.717788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 558/1136] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:42.988157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb3d4-0a67-4478-927d-95cbdb9e3907 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:43.089434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a3eb3d4-0a67-4478-927d-95cbdb9e3907 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:43.099024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:43.132972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:43.133598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:43.133876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:43.134146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] PATH_INFO: `/v3/domains/default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:43.135953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:43.137367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7de16ae-2d62-43a4-982f-c2364980d27c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:43.138978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 579/1137] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:07:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:43.145235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:43.160529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:43.246056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce535942-8a66-40da-8854-b96991dbda06 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:43.280163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:43.280759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:43.281036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:43.281306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:43.281960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:43.283414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e81a5e-75b2-4e95-a6bb-9b5823fa195e None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:43.305347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 559/1138] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:07:43 2022] POST /identity/v3/users => generated 286 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:07:45.223750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a56c-374e-47c9-affc-c7fcd21e2d13 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:45.224318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a56c-374e-47c9-affc-c7fcd21e2d13 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:45.224318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5093a56c-374e-47c9-affc-c7fcd21e2d13 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:45.225650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 580/1139] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:45.231432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:45.231841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:45.231986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:45.250956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:45.252167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:45.314775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:45.322650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 560/1140] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:45 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:45.586270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a56c-374e-47c9-affc-c7fcd21e2d13 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:45.685145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5093a56c-374e-47c9-affc-c7fcd21e2d13 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:45.719112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:45.719733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:45.720010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:45.720279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:45.730639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:45.735457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:45.736652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:45.737935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 581/1141] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:45.742604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:45.840979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257b9572-4253-41f7-b95e-68cc7ee2c3c1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:45.873974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:45.874598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:45.874877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:45.875145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:45.875798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:45.876099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:45.877455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:45.885844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 561/1142] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:45.891820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:45.926872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:45.990333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d9b1f8-c5a6-41fd-99c9-5eb8486a0a3d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:46.023885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:46.024503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:46.024783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:46.025650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] PATH_INFO: `/v3/users/110628d5911f44e4b2904df2097b4cef` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:46.038160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] RBAC: Authorizing `identity:get_user(user_id=110628d5911f44e4b2904df2097b4cef)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:46.039603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:46.041326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 582/1143] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:45 2022] GET /identity/v3/users/110628d5911f44e4b2904df2097b4cef => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:46.047323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:46.107121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:46.149692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0724f6c4-bfcc-428a-b585-878caef8633b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:46.183290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:46.183890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:46.184168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:46.184437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:46.249527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:46.250984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:46.305957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:46.345293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 562/1144] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:46 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:46.351181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:46.444554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:46.452409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b443b1-c3ea-4086-bc88-9ac9ce482fec None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:46.485827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:46.486472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:46.486756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:46.487026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] PATH_INFO: `/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:46.488136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] RBAC: Authorizing `identity:get_project(project_id=77972924dddf4ffc9f6cb70f7d3c94df)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:46.489560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e01a717-fd9c-46ea-946a-6de9be8d2ef4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:46.491200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 583/1145] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:46 2022] GET /identity/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:46.497443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:46.594884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc0fceb-06a4-46d8-8c27-5cc664c4c2a6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:46.626984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:46.627576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:46.627851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:46.628121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:46.628834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=77972924dddf4ffc9f6cb70f7d3c94df, user.id=110628d5911f44e4b2904df2097b4cef) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:46.629135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:46.630582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df21d31-9fed-4645-930f-343e642c3ec7 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:46.636310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 563/1146] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:46 2022] GET /identity/v3/role_assignments?user.id=110628d5911f44e4b2904df2097b4cef&scope.project.id=77972924dddf4ffc9f6cb70f7d3c94df&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:48.502832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b505-3c04-4840-9424-8fda8b7e0895 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:48.503367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b505-3c04-4840-9424-8fda8b7e0895 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:48.503367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6961b505-3c04-4840-9424-8fda8b7e0895 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:48.504710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 584/1147] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:48.509889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:48.510348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:48.510445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:48.529710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:48.530916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:48.547837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:48.597769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 564/1148] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:48 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:48.867929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b505-3c04-4840-9424-8fda8b7e0895 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:48.967296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6961b505-3c04-4840-9424-8fda8b7e0895 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:49.001144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:49.001761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:49.002043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:49.002341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:49.012712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:49.017530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:49.018495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:49.019395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 585/1149] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:49.025599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:49.120730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07833f20-a231-4836-b40a-53e1cc19aef1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:49.153325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:49.153917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:49.154190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:49.154483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:49.155132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:49.155433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:49.156777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:49.164476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 565/1150] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:49.170688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:49.195452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:49.269548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce4920c-2639-4836-b45a-f124efc0f9af None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:49.302099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:49.302755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:49.303037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:49.303310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] PATH_INFO: `/v3/users/110628d5911f44e4b2904df2097b4cef` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:49.304422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] RBAC: Authorizing `identity:get_user(user_id=110628d5911f44e4b2904df2097b4cef)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:49.305837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:49.308082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 586/1151] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:49 2022] GET /identity/v3/users/110628d5911f44e4b2904df2097b4cef => generated 286 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:49.313843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:49.332829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:49.414366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90609311-c2fd-45fe-9a47-e3ea86a0c283 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:49.448293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:49.448909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:49.449189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:49.449460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] PATH_INFO: `/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:49.450601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] RBAC: Authorizing `identity:get_project(project_id=77972924dddf4ffc9f6cb70f7d3c94df)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:49.452026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-909b244e-b07a-4315-a105-c1caac482723 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:49.453654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 566/1152] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:49 2022] GET /identity/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:49.459637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:49.548045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:49.558398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce3b92d-7dda-477c-a75c-7d8a87666981 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:49.590702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:49.591349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:49.591623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:49.591887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] PATH_INFO: `/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df/users/110628d5911f44e4b2904df2097b4cef/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:49.593770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] RBAC: Authorizing `identity:create_grant(project_id=77972924dddf4ffc9f6cb70f7d3c94df, user_id=110628d5911f44e4b2904df2097b4cef, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:49.595205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf3e3b1-8e88-48b9-ac38-b300a80e3d15 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:49.608842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 587/1153] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:07:49 2022] PUT /identity/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df/users/110628d5911f44e4b2904df2097b4cef/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:07:51.527216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff4de-1651-4664-93bb-bb2e1ad25d17 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:51.527793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff4de-1651-4664-93bb-bb2e1ad25d17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:51.528008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566ff4de-1651-4664-93bb-bb2e1ad25d17 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:51.530162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 567/1154] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:51.535411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:51.535805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:51.535949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:51.554161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:51.555359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:51.618522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 588/1155] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:51 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:51.886948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ff4de-1651-4664-93bb-bb2e1ad25d17 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:51.971500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:51.988475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566ff4de-1651-4664-93bb-bb2e1ad25d17 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.022415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.023027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.023304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.023575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.034127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:07:52.038978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:52.040181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:07:52.041594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 568/1156] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:07:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:52.046200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:52.124676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:52.145266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf0f77f-a875-4f9d-a4f3-f85c699b74e6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.178612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.179207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.179488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.179758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.180414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:52.180715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:52.182777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:52.191051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 589/1157] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:07:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:52.197592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:52.293081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d21c409-308a-4883-b6c5-8dd4a6d9c46d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.326564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.327173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.327448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.327718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] PATH_INFO: `/v3/users/110628d5911f44e4b2904df2097b4cef` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.328821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] RBAC: Authorizing `identity:get_user(user_id=110628d5911f44e4b2904df2097b4cef)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:52.330228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:52.331800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 569/1158] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:07:52 2022] GET /identity/v3/users/110628d5911f44e4b2904df2097b4cef => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:52.337788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:52.436004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0457514-3521-4fdc-b68e-0e394c8e2621 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.468366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.468961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.469241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.469510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.481631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:52.537352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:52.538824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:52.635013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 590/1159] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:52 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:52.640991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:52.656864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:52.740598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4d11dd-0073-4a42-9d3b-cd5626c94514 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.773183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.773797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.774074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.774374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] PATH_INFO: `/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.776240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] RBAC: Authorizing `identity:get_project(project_id=77972924dddf4ffc9f6cb70f7d3c94df)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:52.777671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec6a753-e08a-43bc-9cdc-89cb5b88a8ce None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:52.779335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 570/1160] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:07:52 2022] GET /identity/v3/projects/77972924dddf4ffc9f6cb70f7d3c94df => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:52.785491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:52.828853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:52.905380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc37db3-8d75-46c6-ae9e-34be611e322b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:52.945832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:52.946454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:52.946908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:52.947327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:52.948045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=77972924dddf4ffc9f6cb70f7d3c94df, user.id=110628d5911f44e4b2904df2097b4cef) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:52.948349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:52.949753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1201dec4-6b36-48e1-a6fd-da0f63c23441 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:52.955377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 591/1161] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:07:52 2022] GET /identity/v3/role_assignments?user.id=110628d5911f44e4b2904df2097b4cef&scope.project.id=77972924dddf4ffc9f6cb70f7d3c94df&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:54.893220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2d9e-278a-4ea5-b0f1-5501d3ea5aec None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:54.893789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2d9e-278a-4ea5-b0f1-5501d3ea5aec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:54.893789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c2d9e-278a-4ea5-b0f1-5501d3ea5aec None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:54.895171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 571/1162] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:54.902104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:54.902627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:54.902742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:54.923049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:54.924250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:54.989365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 592/1163] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:54 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:55.254696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2d9e-278a-4ea5-b0f1-5501d3ea5aec None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:55.336724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:55.357942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c2d9e-278a-4ea5-b0f1-5501d3ea5aec None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:55.390082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:55.390710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:55.390988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:55.391253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:55.456399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:55.457825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:55.535268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:55.554653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 572/1164] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:07:55 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:07:55.560678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:55.594863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:55.657898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08250e7e-1ae7-48e0-a685-a293b64815d0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:55.690583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:55.691198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:55.691473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:55.691737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:55.698973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:55.700393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5a64ab5c-d9de-46a8-a918-1303c5f1eb02 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:07:55.711705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:07:55.713037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:07:55.714185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 593/1165] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:07:55 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:07:55.718926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:55.819527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1c4f6b-684a-42e7-aeff-df795771253f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:55.853186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:55.853907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:55.854194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:55.854493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:55.855129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:07:55.855429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:55.856782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1627d61a-223c-48a0-9212-39cfc9c84bca None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:55.863701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 573/1166] 10.222.0.46 () {56 vars in 1115 bytes} [Tue Aug 30 04:07:55 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:07:57.762860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e106a-d1bf-47a3-b050-8a33baf44d6c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:57.763428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e106a-d1bf-47a3-b050-8a33baf44d6c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:57.763428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89e106a-d1bf-47a3-b050-8a33baf44d6c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:57.764720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 594/1167] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:07:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:07:57.769872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751a08-6e09-4699-b132-de2379d1ea77 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:57.770281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751a08-6e09-4699-b132-de2379d1ea77 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:57.770460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751a08-6e09-4699-b132-de2379d1ea77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:57.788620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f2751a08-6e09-4699-b132-de2379d1ea77 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:07:57.789806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2751a08-6e09-4699-b132-de2379d1ea77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:07:57.831204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:57.858094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 574/1168] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:07:57 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:07:58.138379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e106a-d1bf-47a3-b050-8a33baf44d6c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:07:58.231773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:07:58.238796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89e106a-d1bf-47a3-b050-8a33baf44d6c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:07:58.272367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:07:58.272959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:07:58.273238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:07:58.273506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:07:58.274173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:07:58.276189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf040be-b49f-4cfe-8d35-df613fe9b30a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:07:58.291819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 595/1169] 10.222.0.46 () {60 vars in 1138 bytes} [Tue Aug 30 04:07:58 2022] POST /identity/v3/domains => generated 267 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:08:00.216111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401b46-f117-44d8-91d2-194da793c8dc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:00.216646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401b46-f117-44d8-91d2-194da793c8dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:00.216646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f401b46-f117-44d8-91d2-194da793c8dc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:00.217999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 575/1170] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:08:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:08:00.223118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:00.223536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:00.223658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:00.241635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:08:00.242882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:08:00.309182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 596/1171] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:08:00 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:08:00.586864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401b46-f117-44d8-91d2-194da793c8dc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:00.688723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f401b46-f117-44d8-91d2-194da793c8dc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:00.722199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:00.722842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:00.723117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:00.723383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] PATH_INFO: `/v3/domains/17d3404272a64296b94c465d82b4f6ff` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:00.733758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] RBAC: Authorizing `identity:get_domain(domain_id=17d3404272a64296b94c465d82b4f6ff)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:00.735257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6563840-188f-4779-85bf-97f8f8874985 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:00.736848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 576/1172] 10.222.0.46 () {56 vars in 1183 bytes} [Tue Aug 30 04:08:00 2022] GET /identity/v3/domains/17d3404272a64296b94c465d82b4f6ff => generated 267 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:00.742731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:00.826972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:00.842648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70f1b3e-01e7-4763-82a4-fb588170246d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:00.878327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:00.878926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:00.879200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:00.879465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:00.880102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:00.881510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed94213-914a-4363-9784-2d93323e69ea None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:00.907608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 597/1173] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:08:00 2022] POST /identity/v3/projects => generated 361 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:08:02.835757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75c2c1-8515-47e4-9d15-4cd819df9921 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:02.836224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75c2c1-8515-47e4-9d15-4cd819df9921 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:02.836324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af75c2c1-8515-47e4-9d15-4cd819df9921 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:02.838631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 577/1174] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:08:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:08:02.843696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:02.844145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:02.844247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:02.863037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:08:02.864219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:08:02.929396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 598/1175] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:08:02 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:08:03.199497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75c2c1-8515-47e4-9d15-4cd819df9921 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:03.234909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:03.300904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af75c2c1-8515-47e4-9d15-4cd819df9921 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:03.337571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:03.338294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:03.338580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:03.338857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] PATH_INFO: `/v3/domains/17d3404272a64296b94c465d82b4f6ff` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:03.340010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] RBAC: Authorizing `identity:get_domain(domain_id=17d3404272a64296b94c465d82b4f6ff)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:03.341448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e245318-e1a5-4409-88a7-a5c19cced700 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:03.343055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 578/1176] 10.222.0.46 () {56 vars in 1183 bytes} [Tue Aug 30 04:08:03 2022] GET /identity/v3/domains/17d3404272a64296b94c465d82b4f6ff => generated 267 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:03.348814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:03.416143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:03.449299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07402ea-ce59-4adc-aa10-d7dba755d182 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:03.483217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:03.483754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:03.484028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:03.484296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:03.484931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:03.486360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfa3c61-2446-456e-ae28-48704d053bee None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:03.506871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 599/1177] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:08:03 2022] POST /identity/v3/users => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:08:05.483124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe5df3-7beb-4f22-bbea-59024b4295a8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:05.483681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe5df3-7beb-4f22-bbea-59024b4295a8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:05.483681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cbe5df3-7beb-4f22-bbea-59024b4295a8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:05.484986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 579/1178] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:08:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:08:05.490021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:05.490434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:05.490604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:05.510100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:08:05.511343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:08:05.576758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 600/1179] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:08:05 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:08:05.846234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe5df3-7beb-4f22-bbea-59024b4295a8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:05.947054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cbe5df3-7beb-4f22-bbea-59024b4295a8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:05.948794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:05.982460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:05.983076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:05.983356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:05.983629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:05.993591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:08:05.998598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:08:05.999763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:08:06.000878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 580/1180] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:08:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:08:06.005673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:06.059735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:06.104118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce754aa3-d3b1-42f6-8ef4-7c353998dba0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:06.137432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:06.138034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:06.138340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:06.138614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:06.139269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:08:06.139573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:06.140930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:06.151448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 601/1181] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:08:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:06.157311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:06.269629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2ef40c-dcb6-4e71-a19c-fae11c663465 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:06.304113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:06.304736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:06.305012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:06.305282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] PATH_INFO: `/v3/users/4275d8830d8e46c29648998da2122862` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:06.318771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] RBAC: Authorizing `identity:get_user(user_id=4275d8830d8e46c29648998da2122862)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:06.320211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:06.322736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 581/1182] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:08:06 2022] GET /identity/v3/users/4275d8830d8e46c29648998da2122862 => generated 311 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:06.328771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:06.426777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44ed1cff-e80b-4c0c-9095-1fd48f1f13fa None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:06.442754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:06.460858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:06.461460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:06.461740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:06.462043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:06.527553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:06.528985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:06.624946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 602/1183] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:08:06 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:08:06.630969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:06.653321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:06.733192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73890639-26dc-4bf6-95d2-6aba8f9d67bd None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:06.771350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:06.771963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:06.772236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:06.772499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] PATH_INFO: `/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:06.773599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] RBAC: Authorizing `identity:get_project(project_id=f7ceb613a9614d4f87dddfbd59e59d3d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:06.775656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5210b5bb-8aeb-4b4d-b832-2a231d273b5b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:06.776826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 582/1184] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:08:06 2022] GET /identity/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:06.783005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:06.800623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:06.880985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25eea805-e85a-4682-afc4-98cf832387d8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:06.913370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:06.913958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:06.914234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:06.914537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:06.915777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=f7ceb613a9614d4f87dddfbd59e59d3d, user.id=4275d8830d8e46c29648998da2122862) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:08:06.916155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:06.917560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56de867-223d-4cd2-aa22-c45d874f9299 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:06.922807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 603/1185] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:08:06 2022] GET /identity/v3/role_assignments?user.id=4275d8830d8e46c29648998da2122862&scope.project.id=f7ceb613a9614d4f87dddfbd59e59d3d&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:08.819780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5410e-c8f8-4900-bc28-5c223b905326 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:08.820217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5410e-c8f8-4900-bc28-5c223b905326 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:08.820313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5410e-c8f8-4900-bc28-5c223b905326 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:08.821796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 583/1186] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:08:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:08:08.827274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:08.827747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:08.827865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:08.845833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:08:08.847034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:08:08.911560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 604/1187] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:08:08 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:08:09.181638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5410e-c8f8-4900-bc28-5c223b905326 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:09.268353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:09.282145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5410e-c8f8-4900-bc28-5c223b905326 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:09.316007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:09.316627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:09.316903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:09.317170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:09.327264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:08:09.332068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:08:09.333161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:08:09.334199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 584/1188] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:08:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:08:09.340028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:09.351407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:09.456786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab905508-ca36-4ac8-a13a-87cf173b1595 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:09.500551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:09.501154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:09.501432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:09.501698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:09.502379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:08:09.502678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:09.504030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:09.512477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 605/1189] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:08:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:09.518456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:09.616707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec705d56-544d-4a4f-9162-cd2151b4a1bb None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:09.651936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba313f76-cc74-4566-a170-65246830769f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:09.652615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313f76-cc74-4566-a170-65246830769f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:09.652895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313f76-cc74-4566-a170-65246830769f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:09.653166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba313f76-cc74-4566-a170-65246830769f None admin] PATH_INFO: `/v3/users/4275d8830d8e46c29648998da2122862` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:09.654311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba313f76-cc74-4566-a170-65246830769f None admin] RBAC: Authorizing `identity:get_user(user_id=4275d8830d8e46c29648998da2122862)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:09.655730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba313f76-cc74-4566-a170-65246830769f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:09.657305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 585/1190] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:08:09 2022] GET /identity/v3/users/4275d8830d8e46c29648998da2122862 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:09.663098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:09.731164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:09.760291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ecf679a-ad38-4cf9-aa85-08647fff0ef3 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:09.802680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:09.803303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:09.803577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:09.803841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] PATH_INFO: `/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:09.804949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] RBAC: Authorizing `identity:get_project(project_id=f7ceb613a9614d4f87dddfbd59e59d3d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:09.807302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bff07b-8d0b-4e24-bc2d-5adcc06e37ac None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:09.809099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 606/1191] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:08:09 2022] GET /identity/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d => generated 361 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:09.815608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba313f76-cc74-4566-a170-65246830769f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:09.836280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:09.924087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba313f76-cc74-4566-a170-65246830769f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:09.956565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:09.957210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:09.957487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:09.957757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] PATH_INFO: `/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d/users/4275d8830d8e46c29648998da2122862/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:09.959654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] RBAC: Authorizing `identity:create_grant(project_id=f7ceb613a9614d4f87dddfbd59e59d3d, user_id=4275d8830d8e46c29648998da2122862, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:09.961062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463efc98-707b-4db1-b140-0f4780dbb8f6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:09.973337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 586/1192] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:08:09 2022] PUT /identity/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d/users/4275d8830d8e46c29648998da2122862/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:08:11.878723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5863-3290-491d-94e1-f3fc3a46fedc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:11.879259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5863-3290-491d-94e1-f3fc3a46fedc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:11.879259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e5863-3290-491d-94e1-f3fc3a46fedc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:11.880578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 607/1193] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:08:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:08:11.886776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:11.887158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:11.887324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:11.905620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:08:11.906842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:08:11.980293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 587/1194] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:08:11 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:08:12.254472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5863-3290-491d-94e1-f3fc3a46fedc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:12.357400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e5863-3290-491d-94e1-f3fc3a46fedc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:12.389545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:12.390154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:12.390467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:12.390738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:12.398284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:12.402611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:08:12.407592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:08:12.408788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:08:12.409851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 608/1195] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:08:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:08:12.414894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:12.423616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:12.513305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba351749-868a-49e6-931f-eecfcf36fe43 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:12.546981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:12.547572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:12.547843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:12.548107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:12.548755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:08:12.549053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:12.550426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:12.558770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 588/1196] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:08:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:12.564704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:12.663175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e421b2-8610-4e6a-a293-a55881d7c0db None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:12.696192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:12.697603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:12.697884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:12.698154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] PATH_INFO: `/v3/users/4275d8830d8e46c29648998da2122862` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:12.699302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] RBAC: Authorizing `identity:get_user(user_id=4275d8830d8e46c29648998da2122862)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:12.700716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:12.702310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 609/1197] 10.222.0.46 () {56 vars in 1177 bytes} [Tue Aug 30 04:08:12 2022] GET /identity/v3/users/4275d8830d8e46c29648998da2122862 => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:12.708421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:12.773708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:12.806015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378b6bfd-26f1-4812-b295-221166da69f3 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:12.840073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:12.840673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:12.840945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:12.841213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:12.905949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:12.907404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:12.968456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:13.000816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 589/1198] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:08:12 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:08:13.007401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:13.077075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:13.110002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae59f59-2096-4473-ab4f-540acc2f556c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:13.143556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:13.144206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:13.144480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:13.144747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] PATH_INFO: `/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:13.146103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] RBAC: Authorizing `identity:get_project(project_id=f7ceb613a9614d4f87dddfbd59e59d3d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:13.147552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cab31d7-b77a-4a73-9fb8-2bfa238d942e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:13.149172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 610/1199] 10.222.0.46 () {56 vars in 1186 bytes} [Tue Aug 30 04:08:13 2022] GET /identity/v3/projects/f7ceb613a9614d4f87dddfbd59e59d3d => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:13.156083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:08:13.255849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc964d6b-0f0d-466f-88cd-26e45c309f64 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:08:13.289182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:08:13.289771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:08:13.290698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:08:13.291009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:08:13.291839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=f7ceb613a9614d4f87dddfbd59e59d3d, user.id=4275d8830d8e46c29648998da2122862) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:08:13.292140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:08:13.293547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09f784d-7882-42c3-8ecd-786bbbc0e77f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:08:13.299143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 590/1200] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:08:13 2022] GET /identity/v3/role_assignments?user.id=4275d8830d8e46c29648998da2122862&scope.project.id=f7ceb613a9614d4f87dddfbd59e59d3d&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:08:23.139363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=32 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:08:23.296569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:09.236351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb75dbc-aef4-4475-8c63-f8cfb4a31a5b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:09.236949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb75dbc-aef4-4475-8c63-f8cfb4a31a5b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:09.236949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb75dbc-aef4-4475-8c63-f8cfb4a31a5b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:09.238221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 611/1201] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:09.248198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:09.248733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:09.248837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:09.270239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:09.271796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:09.361490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 591/1202] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:09 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:09:09.651570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb75dbc-aef4-4475-8c63-f8cfb4a31a5b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:09.786499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb75dbc-aef4-4475-8c63-f8cfb4a31a5b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:09.832907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:09.833671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:09.833921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:09.834294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:09.847603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:09.849172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:09.865956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:09.867165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:09.869135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:09.869135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:09.869135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:09.869135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 612/1203] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:09.875778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:10.010427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e185d-bd79-4866-98ad-4e56018c0570 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:10.045621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:10.052529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:10.053275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:10.053605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:10.053918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:10.054579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:10.054923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:10.057005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc95388-8761-4eff-866f-75f5ac3c7d49 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:10.078356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 592/1204] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:10.078569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:10.146746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:10.222012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a73ac47-6831-4422-8111-9de32d8400b5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:10.263657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:10.264310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:10.264590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:10.264937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:10.265732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:10.267308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395cb74-3357-48e3-bb61-fb308583a0f4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:10.295586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 613/1205] 10.222.0.46 () {60 vars in 1132 bytes} [Tue Aug 30 04:09:10 2022] POST /identity/v3/users => generated 251 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:09:12.524945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd42e8-430a-465a-b6d4-95e7fd7b1888 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:12.525501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd42e8-430a-465a-b6d4-95e7fd7b1888 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:12.525501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fd42e8-430a-465a-b6d4-95e7fd7b1888 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:12.527130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 593/1206] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:12.532758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:12.533179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:12.533381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:12.553367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:12.555432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:12.638843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 614/1207] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:12 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:12.908566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd42e8-430a-465a-b6d4-95e7fd7b1888 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:13.035400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fd42e8-430a-465a-b6d4-95e7fd7b1888 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:13.082117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:13.082831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:13.083140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:13.083517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:13.098031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:13.104685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:13.105857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:13.107118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 594/1208] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:09:12 2022] GET /identity/v3/roles/service => generated 85 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:13.112603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:13.155297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:13.222496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-305b53fd-97a1-4ec0-be0f-d1cd7920db76 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:13.261533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:13.262145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:13.262487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:13.262807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:13.263520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:13.263828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:13.265273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:13.275138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 615/1209] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:09:13 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:13.283660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:13.349096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:13.413229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4e178ea-5355-4c8b-8962-9b9ae883f8dc None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:13.457296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:13.457934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:13.458284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:13.458496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:13.470386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:13.473956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:13.489209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:13.490607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:13.491525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:13.491525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:13.491525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:13.491525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 595/1210] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:13.498427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:13.613097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:13.617162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e1399c-556f-40f8-9514-d4e899c92e1d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:13.661647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:13.662444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:13.662860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:13.663193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:13.663937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:13.664355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:13.666088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:13.683548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 616/1211] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:13.689338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:13.806314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a8112c-c869-4e8d-b478-a89bd9e767d5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:13.809289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:13.848025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:13.848786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:13.849037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:13.849370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:13.859570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:13.861069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:13.874321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:13.875308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:13.876273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 596/1212] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:13 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:13.881852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:14.000237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf0ced5-1238-445a-ae6d-1ac8c07bf2e8 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:14.041682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:14.042404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:14.042595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:14.042893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:14.043708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:14.044006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:14.045621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:14.066003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 617/1213] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:13 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:14.072691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:14.201997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377378ab-7434-4fb3-95a3-c97fc1531023 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:14.247780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:14.248395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:14.248672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:14.249068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:14.288903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:14.331976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:14.333607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:14.458688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 597/1214] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:09:14 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:09:14.465145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:14.487580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:14.589714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb9478c-2b20-4307-ac2f-d38b43ff94c5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:14.635810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:14.637013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:14.637113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:14.637402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:14.652321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:14.654346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:14.672055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:14.673274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:14.675836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:14.675836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:14.675836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:14.678554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 618/1215] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:14.679444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:14.755305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:14.800068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf8fbe4-2ccc-456a-a727-e0e8d748a87c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:14.845502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:14.846128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:14.846463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:14.846827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:14.847365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:14.847669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:14.849119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:14.862608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 598/1216] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:14.868198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:14.959900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:14.995321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b03eb0-a841-4e13-88f0-9927609749d5 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:15.032388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:15.032885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:15.033193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:15.033417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:15.040481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:15.041904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:15.058569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:15.059692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:15.060641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:15.065494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 619/1217] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:14 2022] GET /identity/v3/projects/service => generated 88 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:15.065635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:15.182687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0435e8-b6f1-4d5d-bf83-1088c4b42103 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:15.208396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:15.223098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:15.223948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:15.224315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:15.224795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:15.225595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:15.226116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:15.228858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b1035de-9c88-4d35-9a82-23b3e326f949 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:15.247951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 599/1218] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:15.249774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:15.363672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2618f9-9f97-4717-be2c-f703a820ac7a None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:15.393527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:15.400800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:15.401430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:15.401698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:15.401967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:15.402720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=ba39d57469d34f258aa903bff3935c3f) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:15.403028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:15.404448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a1bf4e-d5e9-42b7-8183-3de95c04fd52 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:15.410374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 620/1219] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:09:15 2022] GET /identity/v3/role_assignments?user.id=ba39d57469d34f258aa903bff3935c3f&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:17.430855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf5e7-5572-477a-86b3-acd16ea52be3 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:17.431639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf5e7-5572-477a-86b3-acd16ea52be3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:17.431639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bdf5e7-5572-477a-86b3-acd16ea52be3 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:17.432809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 600/1220] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:17.438167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:17.438707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:17.438809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:17.457583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:17.458824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:17.533816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 621/1221] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:17 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:17.827197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdf5e7-5572-477a-86b3-acd16ea52be3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:17.943515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bdf5e7-5572-477a-86b3-acd16ea52be3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:17.980741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:17.981377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:17.981644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:17.981948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] PATH_INFO: `/v3/roles/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:17.992743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:17.998023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:17.999465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:18.000341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 601/1222] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:09:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:18.005594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:18.109023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4d9e2e-81a5-4a94-86b9-74d5f5201832 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:18.136874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:18.152715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:18.153397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:18.153675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:18.154000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:18.154813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] RBAC: Adding query filter params (name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:18.155131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:18.156835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:18.167931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 622/1223] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:09:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:18.174050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:18.232878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:18.299505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4013a449-901d-4dd2-b473-8ce59e22a20e None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:18.335672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:18.337351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:18.337721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:18.338026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:18.354834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:18.356336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:18.375134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:18.376321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:18.377579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:18.377579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:18.377579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:18.377579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 602/1224] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:18.382105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:18.509194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800a9921-67ac-46a8-8876-1d6f4a66a982 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:18.553969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:18.555512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:18.555953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:18.556275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:18.556971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:18.557275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:18.558764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:18.574279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 623/1225] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:18.578624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:18.708156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:18.713075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665ed27-ea8d-47e2-ab1e-26b0f57f35c5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:18.756959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:18.757712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:18.758073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:18.758514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:18.767603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:18.769362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:18.783636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:18.785089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:18.786346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:18.787545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 603/1226] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:18 2022] GET /identity/v3/users/placement => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:18.795099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:18.861546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:18.929134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a6205c-1c99-413f-921a-20d2182aeac2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:18.974338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:18.974917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:18.975175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:18.975438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:18.976174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:18.976480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:18.978221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:19.000577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 624/1227] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:18 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:19.010923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:19.146982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a754b8e-106c-4844-bf93-5960a90e1843 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:19.203827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:19.204941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:19.205459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:19.205971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:19.220690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:19.222156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:19.238126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:19.239372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:19.240432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:19.240432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:19.240432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:19.240432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 604/1228] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:19.245982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:19.362634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71f21dd-97a3-4fa5-8413-a230caf4dd9c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:19.365010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:19.403889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:19.404495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:19.404772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:19.405063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:19.405753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:19.406058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:19.407610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:19.420255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 625/1229] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:19.426194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:19.435849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:19.535419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097a48da-faf5-461e-ba46-6cdce9847497 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:19.574389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:19.576676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:19.576676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:19.576676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:19.583335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:19.585009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:19.598900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:19.599953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:19.601030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 605/1230] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:19.607369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:19.735670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a382fab3-2e42-477c-aa81-fdf192bb879b None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:19.786163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:19.786990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:19.787375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:19.787865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:19.788878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:19.789994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:19.791721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cc593cb-0bfc-4c49-94eb-1ffa98d7cc4e None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:19.807450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 626/1231] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:09:19.813230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:19.889881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:19.928753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b99b03-d4c7-4178-9c92-6cb6738769ef None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:19.974046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:19.974733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:19.975052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:19.975326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/ba39d57469d34f258aa903bff3935c3f/roles/05f2173f893f49a3b546f05eaf0e367f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:19.991600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=ba39d57469d34f258aa903bff3935c3f, role_id=05f2173f893f49a3b546f05eaf0e367f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:19.994099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592b24cf-39e5-4a15-9013-7620e4d04de5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:20.022513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 606/1232] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:09:19 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/ba39d57469d34f258aa903bff3935c3f/roles/05f2173f893f49a3b546f05eaf0e367f => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:09:21.953834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c24e-bcc4-406f-80d9-d1fd9b008b37 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:21.954393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c24e-bcc4-406f-80d9-d1fd9b008b37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:21.954393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af19c24e-bcc4-406f-80d9-d1fd9b008b37 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:21.955764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 627/1233] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:21.961025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:21.962234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:21.962455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:21.980643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:21.981943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:22.051930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 607/1234] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:21 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:22.331346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c24e-bcc4-406f-80d9-d1fd9b008b37 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:22.358655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:22.439866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af19c24e-bcc4-406f-80d9-d1fd9b008b37 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:22.477793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:22.478453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:22.478738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:22.479003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] PATH_INFO: `/v3/roles/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:22.489883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:22.494800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:22.495958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:22.496868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 628/1235] 10.222.0.46 () {56 vars in 1102 bytes} [Tue Aug 30 04:09:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:22.502014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:22.557911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:22.619914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2972f8-21ea-4ea5-929b-9132ce78ec8b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:22.660511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:22.661140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:22.661448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:22.661731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:22.662633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] RBAC: Adding query filter params (name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:22.663002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:22.665509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:22.675191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 608/1236] 10.222.0.46 () {56 vars in 1103 bytes} [Tue Aug 30 04:09:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:22.684732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:22.784272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:22.800747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc4afa7-a67b-4337-b887-0ffd58a35273 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:22.837311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:22.837944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:22.838224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:22.838531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:22.850080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:22.851599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:22.867082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:22.868946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:22.875162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:22.875162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:22.875162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:22.875162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 629/1237] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:22.875478 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:22.978976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7c29cf-dad9-41fc-b488-b8ccb097cbb1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:22.989031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:23.018878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:23.019487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:23.019756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:23.020020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:23.020657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:23.021217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:23.022592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:23.035347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 609/1238] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:23.041324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:23.154849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a90e0f-b91a-4880-968d-995368a27b3d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:23.193989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:23.194945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:23.196099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:23.196324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:23.200160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:23.208659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:23.210739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:23.228087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:23.229343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:23.230548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 630/1239] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:23 2022] GET /identity/v3/users/placement => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:23.235484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:23.350075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8e1dd3-3fc5-4fb7-8c5c-578596b1138c None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:23.392232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:23.392841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:23.393119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:23.393409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:23.394126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:23.394464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:23.395895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:23.415861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 610/1240] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:23 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:23.421667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:23.533653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ad7256-3d21-46b8-8c0b-edc5eb4d13a4 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:23.572576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:23.573183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:23.573461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:23.573726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:23.643745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:23.652606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:23.654048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:23.770889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 631/1241] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:09:23 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:09:23.777739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:23.813112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:23.894735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc83983-6159-46cb-82be-b8a99dd13171 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:23.931422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:23.932280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:23.932547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:23.932818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:23.942791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:23.944287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:23.959537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:23.960728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:23.963068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:23.963068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:23.963068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:23.963068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 611/1242] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:23.967461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:24.073205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:24.080938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9738ed39-8f73-4771-b948-db24583253b7 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:24.115779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:24.116384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:24.116661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:24.116929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:24.117827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:24.118344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:24.119704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:24.133966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 632/1243] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:24.140220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:24.239889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:24.266024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfdeeb7-8a31-43f3-8678-33434f2cbce6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:24.303659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:24.304344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:24.304629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:24.305490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:24.312744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:24.314322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:24.326609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:24.327525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:24.328459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 612/1244] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:09:24.333773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:24.455271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d4221b4-2355-4553-91b9-015de51887c1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:24.492905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:24.493525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:24.493801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:24.494081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:24.494833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:24.495153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:24.496598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5c4a68-d14e-44c5-8424-6b2d7e9fdc60 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:24.509287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 633/1245] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:24.515775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:24.627056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-019c5e4d-9303-4ff0-b0f9-fbba023e1ce2 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:24.668755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:24.669361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:24.669639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:24.669905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:24.673283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] RBAC: Adding query filter params (role.id=05f2173f893f49a3b546f05eaf0e367f, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=ba39d57469d34f258aa903bff3935c3f) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:24.673600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:24.675123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c5f08d-2f33-474e-8bb0-651ba699a220 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:24.682222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 613/1246] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:09:24 2022] GET /identity/v3/role_assignments?user.id=ba39d57469d34f258aa903bff3935c3f&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=05f2173f893f49a3b546f05eaf0e367f => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:26.782556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed7f9-0fc0-4842-b822-e2049f848e89 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:26.783138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed7f9-0fc0-4842-b822-e2049f848e89 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:26.783138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed7f9-0fc0-4842-b822-e2049f848e89 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:26.786282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 634/1247] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:26.790934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:26.791421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:26.791529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:26.798704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:26.815474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:26.816675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:26.893110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 614/1248] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:26 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Aug 30 04:09:27.172450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed7f9-0fc0-4842-b822-e2049f848e89 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:27.197786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:27.301854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed7f9-0fc0-4842-b822-e2049f848e89 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:27.341987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:27.342650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:27.342954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:27.343226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:27.355020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:27.360182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:27.361357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:27.362609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 635/1249] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:27.368809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:27.498797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51562d1e-2318-4981-9aff-2ba93c25ae53 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:27.536770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:27.543024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:27.543635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:27.543921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:27.544176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:27.544846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:27.545204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:27.546757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:27.555748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 615/1250] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:27.561496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:27.649465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:27.674696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39c32f9-7911-4213-b907-8be4ed651aad None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:27.712388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:27.713043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:27.713324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:27.713591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:27.724662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:27.726362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:27.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:27.745817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:27.750765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:27.750765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:27.750765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:27.750765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 636/1251] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:27.751680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:27.872643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0944c9-1518-45db-a494-d6f13487463f None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:27.918057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:27.918714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:27.918997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:27.919267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:27.919917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:27.920221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:27.921595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:27.936453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 616/1252] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:27.942310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:28.051179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cae13c1-cb32-4096-a479-16960982c6cc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:28.085540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:28.097055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:28.097818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:28.098164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:28.098468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:28.108560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:28.110102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:28.122696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:28.123911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:28.124938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 637/1253] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:28.129926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:28.207891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:28.264174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a7bed7-d32b-41f9-9460-f66e92a86502 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:28.303992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:28.304561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:28.305620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:28.305793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:28.306755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:28.307176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:28.308776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:28.329168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 617/1254] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:28.335079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:28.451463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee85fe01-0319-4254-b4a9-623d8a09f379 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:28.488965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:28.489576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:28.489851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:28.490117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:28.548815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:28.563233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:28.564754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:28.677232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 638/1255] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:09:28 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:09:28.684145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:28.790571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67e5705-a7ac-4bcc-936b-b83adbc02315 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:28.808031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:28.832839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:28.833478 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:28.833756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:28.834028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:28.845085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:28.846571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:28.863906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:28.864943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:28.865863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:28.865863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:28.865863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:28.865863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 618/1256] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:28.870985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:28.973183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:28.987693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7453b75-8dca-471c-bc54-2cbbc05421dc None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:29.026126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:29.026752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:29.027025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:29.027288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:29.027921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:29.028214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:29.029557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:29.042992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 639/1257] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:29.049237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:29.165009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0456f068-772b-4ff4-97c7-2321a5b6b637 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:29.208004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:29.208795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:29.209073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:29.209318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:29.217964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:29.219492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:29.232711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:29.233982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:29.234966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 619/1258] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:09:29.239982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:29.352679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d473ca-66ae-414f-9639-47d8c2ed5b3c None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:29.396646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:29.397245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:29.397520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:29.397782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:29.400030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:29.400030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:29.400308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a4d7e-87f1-4eba-984d-112980187f94 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:29.412854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 640/1259] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:29.420376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:29.448896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:29.540353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4782010a-3bb7-4ec4-bf4c-b14103e9d531 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:29.580907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:29.581527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:29.581801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:29.582067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:29.582846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=ba39d57469d34f258aa903bff3935c3f) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:29.583149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:29.584606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009e3460-113f-4a11-b560-671f822eff02 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:29.591407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 620/1260] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:09:29 2022] GET /identity/v3/role_assignments?user.id=ba39d57469d34f258aa903bff3935c3f&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:31.517079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abf61f-eeca-4605-913f-533a4e086f59 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:31.517611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abf61f-eeca-4605-913f-533a4e086f59 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:31.517611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4abf61f-eeca-4605-913f-533a4e086f59 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:31.518981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 641/1261] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:31.525127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:31.525538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:31.525662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:31.543593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:31.544788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:31.609896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 621/1262] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:31 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:31.874274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abf61f-eeca-4605-913f-533a4e086f59 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:31.886040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:31.980144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4abf61f-eeca-4605-913f-533a4e086f59 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.013398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.014011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.014310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.014578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.024500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:32.029231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:32.030404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:32.031560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 642/1263] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:32.036317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.054398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:32.135237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ec0489-7deb-4b94-b0e4-8883e6c318bf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.172196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.172807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.173077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.173353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.174193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:32.174627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.176134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:32.185490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 622/1264] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:32.192052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.266725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:32.300089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efe02a4-8fd7-419a-adae-9c997b3e1fe1 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.338069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.338764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.339082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.339394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.349725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.351187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:32.368103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:32.369411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:32.370559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:32.370559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:32.370559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:32.370559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 643/1265] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:32.375099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.455820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:32.475784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1187785-9392-46a8-88ca-4419992f83a5 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.509346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.509946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.510235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.510539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.511183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:32.511486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.512850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:32.525492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 623/1266] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:32.531283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.630421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25e3ff0-663c-406c-996a-f9fb97e5e667 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.650418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:32.664858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.665474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.665744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.666010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] PATH_INFO: `/v3/users/placement` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.673220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.674649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:32.686236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:32.687308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:32.688435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 644/1267] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:09:32.693367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.791232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e237803c-8284-4bc6-85a6-90245b668cc4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.824695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.825286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.825557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.825823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.826572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:32.826871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.828278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:32.837396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:32.845651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 624/1268] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:32.851329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:32.952244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6e0d61-8929-451d-8aa2-ebfb5bca6fae None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:32.985084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:32.985699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:32.985975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:32.986242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:32.997330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:32.998804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:33.013347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:33.019037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:33.020221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:33.020221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:33.020221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:33.020221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 645/1269] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:33.020514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:33.119006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25531150-b5e3-44ef-ad4a-e262d50417ba None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:33.153472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:33.154080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:33.154390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:33.154662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:33.155308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:33.155609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:33.157027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:33.169292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 625/1270] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:33.175917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:33.208608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:33.278640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e67aa41-1975-4dde-a15e-fef6e7e4c12d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:33.311326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:33.311946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:33.312224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:33.312494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:33.318843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:33.320262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:33.331184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:33.332353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:33.333727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 646/1271] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Aug 30 04:09:33.338796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:33.395746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:33.438331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34a1c34-5e9c-4aed-8aee-6d5a3d295f5d None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:33.471623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:33.472220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:33.472499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:33.472767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:33.473452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:33.473757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:33.475200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9816df-04d7-49a7-8d14-63c4a5832633 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:33.487963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 626/1272] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:33.494133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:33.585014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:33.594651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c902b55-f998-4c1d-b617-59367999afbb None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:33.627750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:33.628390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:33.628667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:33.628935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] PATH_INFO: `/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/ba39d57469d34f258aa903bff3935c3f/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:33.630826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] RBAC: Authorizing `identity:create_grant(project_id=13e34a5a11cc4678908121bcb19d9359, user_id=ba39d57469d34f258aa903bff3935c3f, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:33.632244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21edaee3-6a8c-4a0f-9ceb-5991e0098cbb None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:33.644911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 647/1273] 10.222.0.46 () {58 vars in 1439 bytes} [Tue Aug 30 04:09:33 2022] PUT /identity/v3/projects/13e34a5a11cc4678908121bcb19d9359/users/ba39d57469d34f258aa903bff3935c3f/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:09:35.546313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56532-4a88-4641-9ce6-2cd777f1b3cc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:35.546843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56532-4a88-4641-9ce6-2cd777f1b3cc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:35.546843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea56532-4a88-4641-9ce6-2cd777f1b3cc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:35.548204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 627/1274] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:35.553309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:35.553723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:35.553851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:35.573170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:35.574424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:35.646662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 648/1275] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:35 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:35.913585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56532-4a88-4641-9ce6-2cd777f1b3cc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.015749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea56532-4a88-4641-9ce6-2cd777f1b3cc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.033234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:36.049782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.050430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.050705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.050971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.060883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(role_id) Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Aug 30 04:09:36.065676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:36.066854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:36.067871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 628/1276] 10.222.0.46 () {56 vars in 1096 bytes} [Tue Aug 30 04:09:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:36.073439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.153141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:36.176585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4b0487-dcc7-49f7-a2a9-f7b6a1785bc8 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.219830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.220785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.221245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.221703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.222880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] RBAC: Adding query filter params (name=admin) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:36.223391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:36.225790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:36.236697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 649/1277] 10.222.0.46 () {56 vars in 1099 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:36.243278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.342146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111c069f-ecf8-43f5-934e-e80e95fc00e4 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.375067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.375686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.375962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.376231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.385948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:36.387421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:36.403217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:36.404388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:36.405371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:36.405371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:36.405371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:36.405371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 629/1278] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:36.410430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.512704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333b4130-3bd7-4bdc-83c1-8e6acad1c5da None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.547166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.547772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.548050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.548320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.548955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:36.549258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:36.550641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:36.563081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 650/1279] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:36.568800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.599257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:36.667611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5499d7e6-c91d-42a8-80e9-54d74c437550 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.700734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.701347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.701619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.701884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] PATH_INFO: `/v3/users/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.709012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:36.710449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_user(user_id) Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Aug 30 04:09:36.721369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:36.722579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:36.723772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 630/1280] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:36.729550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.733377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:36.834473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1cc46b3-9d46-41b3-a88d-4b81ab8eff30 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:36.873092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:36.873613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:36.873893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:36.874164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:36.874925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:36.875230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:36.876643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:36.894532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 651/1281] 10.222.0.46 () {56 vars in 1143 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:36.901413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:36.989315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:36.999407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff36daf3-ccf7-440d-ba4f-69b7305d43c6 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.032823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.033416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.033693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.033961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.099042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.100480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.189225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:37.200085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 631/1282] 10.222.0.46 () {58 vars in 1282 bytes} [Tue Aug 30 04:09:36 2022] GET /identity/v3/auth/tokens => generated 2118 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Aug 30 04:09:37.206060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:37.268601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:37.308120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbf5e3a-7d89-4227-bf46-9f65dea02ff2 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.341853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.342501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.342782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.343052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.352966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.354448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Aug 30 04:09:37.369171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:37.370346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Aug 30 04:09:37.371564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Aug 30 04:09:37.371564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Aug 30 04:09:37.371564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:37.371564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 652/1283] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:09:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:37.376144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:37.473283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1a1928-4760-4e8d-a218-ac81e410bd9b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.506822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.507415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.507690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.508086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.508724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:37.509024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.510459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.521468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 632/1284] 10.222.0.46 () {56 vars in 1109 bytes} [Tue Aug 30 04:09:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:37.527489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:37.626019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0880e7db-f533-4e71-9f69-30a94fc34241 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.639258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:37.659218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.659831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.660109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.660380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.667635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.669055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(project_id) Aug 30 04:09:37.680835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:37.681952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: ERROR keystone.server.flask.application Aug 30 04:09:37.683157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 653/1285] 10.222.0.46 () {56 vars in 1111 bytes} [Tue Aug 30 04:09:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:37.689860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:37.739149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:37.791642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1230051e-dc35-456a-bdd3-b90b0d9cc987 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.825281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.825880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.826154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.826448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.827128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:37.827428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.828837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ced62b07-2747-4f05-acb2-e159c99248e2 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.840677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 633/1286] 10.222.0.46 () {56 vars in 1148 bytes} [Tue Aug 30 04:09:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:37.846731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:37.942831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad3ae32-4b06-44cc-869c-af820ffc0b35 None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:37.974980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:37.975563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:37.975839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:37.976106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:37.976962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] RBAC: Adding query filter params (role.id=6476749a9b1d4b73b7942cf1ab708d3c, scope.project.id=13e34a5a11cc4678908121bcb19d9359, user.id=ba39d57469d34f258aa903bff3935c3f) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:37.977262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:37.978703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380c879-ef54-40d6-8e5f-dbe786d62f8d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:37.984232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 654/1287] 10.222.0.46 () {56 vars in 1374 bytes} [Tue Aug 30 04:09:37 2022] GET /identity/v3/role_assignments?user.id=ba39d57469d34f258aa903bff3935c3f&scope.project.id=13e34a5a11cc4678908121bcb19d9359&role.id=6476749a9b1d4b73b7942cf1ab708d3c => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:39.946742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237273-fb5d-4427-b253-d2aef694760c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:39.947290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237273-fb5d-4427-b253-d2aef694760c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:39.947290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d237273-fb5d-4427-b253-d2aef694760c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:39.948633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 634/1288] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:39.953756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:39.954187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:39.954317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:39.972815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:39.973993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:40.002744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:40.041324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 655/1289] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:39 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:40.317465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d237273-fb5d-4427-b253-d2aef694760c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:40.419235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d237273-fb5d-4427-b253-d2aef694760c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:40.429927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:40.453728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:40.454378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:40.454659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:40.454929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:40.455503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:40.456857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:40.465630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:40.466916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:40.467957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 635/1290] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:09:40 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:40.472757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:40.574479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e3178f-cfe7-42bf-b9b4-eeac559d0e80 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:40.609837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:40.610469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:40.610752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:40.611025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:40.611653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:40.611955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:40.613319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cc7dcc-d64f-4fba-bbff-9dfba880e4db None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:40.618548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 656/1291] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 30 04:09:40 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:40.624107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:40.723622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b58010-5280-4919-bb5f-6b30df9b89a1 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:40.756236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:40.756864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:40.757144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:40.757412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] PATH_INFO: `/v3/services` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:40.758638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] RBAC: Adding query filter params (type=placement) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:40.758942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:40.760300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f47ddf-16ee-4150-996a-ccdf702b25fc None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:40.764632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 636/1292] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 30 04:09:40 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:42.667047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf67f-c680-4764-a0e0-73b846a28aa7 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:42.667610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf67f-c680-4764-a0e0-73b846a28aa7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:42.667610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a8cf67f-c680-4764-a0e0-73b846a28aa7 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:42.668925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 657/1293] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:42.674115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ed9a-78f1-40c5-8206-0af0f3f8cd73 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:42.674552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ed9a-78f1-40c5-8206-0af0f3f8cd73 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:42.674688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f1ed9a-78f1-40c5-8206-0af0f3f8cd73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:42.692418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66f1ed9a-78f1-40c5-8206-0af0f3f8cd73 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:42.693589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66f1ed9a-78f1-40c5-8206-0af0f3f8cd73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:42.722807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:42.759130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 637/1294] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:42 2022] POST /identity/v3/auth/tokens => generated 2118 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:43.051703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf67f-c680-4764-a0e0-73b846a28aa7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:43.082949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:43.154063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a8cf67f-c680-4764-a0e0-73b846a28aa7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:43.191668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:43.192315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:43.192833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:43.193098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:43.194097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] RBAC: Authorizing `identity:create_service()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:43.196612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908cef35-ee95-4974-b0f2-8643c7eba1ff None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:43.213375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 658/1295] 10.222.0.46 () {60 vars in 1141 bytes} [Tue Aug 30 04:09:43 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:09:45.149223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96482393-e791-47ce-ab2f-15eb7b64e57b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:45.149752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96482393-e791-47ce-ab2f-15eb7b64e57b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:45.149752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96482393-e791-47ce-ab2f-15eb7b64e57b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:45.151126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 638/1296] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:45.156676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:45.157245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:45.157245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:45.175900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:45.177072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:45.251300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 659/1297] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:45 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:45.527349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96482393-e791-47ce-ab2f-15eb7b64e57b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:45.634237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96482393-e791-47ce-ab2f-15eb7b64e57b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:45.668799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:45.669420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:45.669698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:45.669972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:45.670577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:45.671944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:45.680899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:45.682066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:45.683305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 639/1298] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:09:45 2022] GET /identity/v3/services/placement => generated 90 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:45.688039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:45.706707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:45.788644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2589b23-5b14-43a7-af43-aa0f780097d4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:45.823256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:45.823856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:45.824130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:45.824397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:45.825017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:45.825312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:45.826696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:45.831658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 660/1299] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 30 04:09:45 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:45.838296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:45.869645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:45.938726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3009277d-ee94-47cf-a8de-4e8d48a8ed5b None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:45.972261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:45.972856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:45.973135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:45.973408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:45.974104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] RBAC: Adding query filter params (interface=public, service_id=7f5758a62dfe4f37ba950d4160f68bc7, region_id=RegionOne) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:45.974457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:45.975872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64b279b-8ce1-45be-aeb3-034792a01608 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:45.981098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 640/1300] 10.222.0.46 () {56 vars in 1251 bytes} [Tue Aug 30 04:09:45 2022] GET /identity/v3/endpoints?service_id=7f5758a62dfe4f37ba950d4160f68bc7&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:45.986894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:46.065363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:46.083520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b7204ce-9d39-400b-b807-4765088fff6d None admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:46.121015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:46.121612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:46.121885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:46.122149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:46.122776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:46.124120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbb4be1-18aa-4cd4-a8e4-415ab6496e34 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:46.130720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 661/1301] 10.222.0.46 () {56 vars in 1087 bytes} [Tue Aug 30 04:09:45 2022] GET /identity/v3/services => generated 2191 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:09:48.053631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d896cff-3675-48c3-844b-6b5f08e7e2f6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:48.054186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d896cff-3675-48c3-844b-6b5f08e7e2f6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:48.054186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d896cff-3675-48c3-844b-6b5f08e7e2f6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:48.056130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 641/1302] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:09:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:09:48.061317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:48.061716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:48.061888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:48.079944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:09:48.081060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:09:48.147675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 662/1303] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:09:48 2022] POST /identity/v3/auth/tokens => generated 2221 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Aug 30 04:09:48.420073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d896cff-3675-48c3-844b-6b5f08e7e2f6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:48.514074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:48.521737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d896cff-3675-48c3-844b-6b5f08e7e2f6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:48.556454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:48.557072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:48.557350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:48.557622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] PATH_INFO: `/v3/services/placement` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:48.558196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:48.559583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Traceback (most recent call last): Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1820, in full_dispatch_request Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application rv = self.dispatch_request() Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1796, in dispatch_request Aug 30 04:09:48.568226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 107, in view Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(**kwargs) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._get_service(service_id) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Aug 30 04:09:48.569405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.get_or_create( Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application with Lock( Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self._enter() Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application return self.creator() Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application created_value = creator( Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: ERROR keystone.server.flask.application Aug 30 04:09:48.570722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 642/1304] 10.222.0.46 () {56 vars in 1117 bytes} [Tue Aug 30 04:09:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Aug 30 04:09:48.575843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:48.652560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:48.676050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6088425-b853-46d5-a158-1ad65bce8768 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:48.709395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:48.709991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:48.710290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:48.710565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] PATH_INFO: `/v3/services` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:48.711563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] RBAC: Adding query filter params (name=placement) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:09:48.711875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:48.713234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dde56c-1fdb-4608-b96e-1e9d20202c90 None admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:48.718192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 663/1305] 10.222.0.46 () {56 vars in 1116 bytes} [Tue Aug 30 04:09:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:09:48.723905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:09:48.824501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3991acb0-f305-4269-98ae-ee6287f01e49 None admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:09:48.857863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:09:48.858496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:09:48.858778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:09:48.859047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] PATH_INFO: `/v3/endpoints` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:09:48.859701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:09:48.861076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d3e96f-cc49-4e8b-b3a8-ccc16c53a18b None admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:09:48.880725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 643/1306] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:09:48 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:09:58.715278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=30 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:09:58.876513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:10:53.360053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5c5e-d140-4829-abe7-ed0943891c2c None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:53.360513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5c5e-d140-4829-abe7-ed0943891c2c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:53.360593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8ba5c5e-d140-4829-abe7-ed0943891c2c None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:53.361890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 664/1307] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:10:53.367589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6dd5a-af2e-42ea-af57-35e0753c5831 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:53.368099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6dd5a-af2e-42ea-af57-35e0753c5831 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:53.368456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a6dd5a-af2e-42ea-af57-35e0753c5831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:53.448638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f2a6dd5a-af2e-42ea-af57-35e0753c5831 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:10:53.449891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f2a6dd5a-af2e-42ea-af57-35e0753c5831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:53.590741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 644/1308] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:10:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:10:53.880144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:53.880510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:53.880692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:53.882305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 665/1309] 10.222.0.46 () {54 vars in 965 bytes} [Tue Aug 30 04:10:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:10:53.889025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6d275-f843-4b0d-8380-1f004691edfd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:53.889315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6d275-f843-4b0d-8380-1f004691edfd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:53.889593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac6d275-f843-4b0d-8380-1f004691edfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:53.975759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1ac6d275-f843-4b0d-8380-1f004691edfd None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:10:53.977818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1ac6d275-f843-4b0d-8380-1f004691edfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:54.000907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:10:54.066459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 645/1310] 10.222.0.46 () {58 vars in 1062 bytes} [Tue Aug 30 04:10:53 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:10:54.076518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:10:54.078629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:54.216529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5610e59d-1b0c-423d-9a5d-9c38324a3676 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:10:54.308179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:10:54.308806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:54.309082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:54.309345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:54.311026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:54.375593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:10:54.439307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:10:54.440753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b67f0f9-20dd-4e72-8cbf-22ffe6fdf563 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:10:54.462158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 666/1311] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:10:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:10:56.739273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:56.739789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:56.739789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:56.741090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 646/1312] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:10:56.746953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f2af-40c1-46af-956f-8fe96f2b8ea6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:56.747385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f2af-40c1-46af-956f-8fe96f2b8ea6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:56.747499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0420f2af-40c1-46af-956f-8fe96f2b8ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:56.841325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0420f2af-40c1-46af-956f-8fe96f2b8ea6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:10:56.842599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0420f2af-40c1-46af-956f-8fe96f2b8ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:56.953643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 667/1313] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:10:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:10:57.235506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:10:57.237019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:57.343100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7b15dd-447c-431e-9cfb-7eeddc32e7dc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:10:57.413246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:10:57.416332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:10:57.417109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:57.417593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:57.418057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:57.419690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:57.526224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:10:57.527596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d92c0eb-808d-4aa8-b493-4738e47fbd71 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:10:57.545499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 647/1314] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:10:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:10:59.772624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:59.773733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:59.773836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:59.775156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 668/1315] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:10:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:10:59.780297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb0ef-b851-4977-a726-8b216244c0c2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:10:59.780707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb0ef-b851-4977-a726-8b216244c0c2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:10:59.780819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ecb0ef-b851-4977-a726-8b216244c0c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:10:59.853174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f8ecb0ef-b851-4977-a726-8b216244c0c2 None None] MFA Rules not processed for user `0674c5f80c3443de9ee43f655c58e809`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:10:59.854489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f8ecb0ef-b851-4977-a726-8b216244c0c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:10:59.906638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:10:59.952466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 648/1316] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:10:59 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:00.241410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:00.243150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:00.252866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:00.348648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-494daa0e-afb6-407b-97e7-109d3545c1da None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:00.421915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:00.422540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:00.422820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:00.423088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:00.424696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:00.495441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:00.557896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:00.559341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c06d5164-7f40-40a6-986b-c79dabacc342 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:00.577448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 669/1317] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:00 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:03.113377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:03.113907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:03.113907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:03.115296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 649/1318] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:03.121427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c595630-e2fe-41c8-a28b-ff73294157a1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:03.121855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c595630-e2fe-41c8-a28b-ff73294157a1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:03.121961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c595630-e2fe-41c8-a28b-ff73294157a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:03.188151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4c595630-e2fe-41c8-a28b-ff73294157a1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:03.189367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c595630-e2fe-41c8-a28b-ff73294157a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:03.286831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 670/1319] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:03.578818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:03.580398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:03.684244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f397f2-386b-470d-991c-ec48b05a9171 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:03.757019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:03.757743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:03.758053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:03.758353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:03.759997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:03.767750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:03.870558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:03.871989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae227a-7f03-4331-94d7-6717b94f1502 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:03.890561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 650/1320] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:03 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:06.469302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:06.469926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:06.469926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:06.471381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 671/1321] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:06.476925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd05e5-2645-4b89-b607-fd1bb35a7cfc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:06.477345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd05e5-2645-4b89-b607-fd1bb35a7cfc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:06.477482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd05e5-2645-4b89-b607-fd1bb35a7cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:06.549423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d9cd05e5-2645-4b89-b607-fd1bb35a7cfc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:06.551416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d9cd05e5-2645-4b89-b607-fd1bb35a7cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:06.646154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:06.716927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 651/1322] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:07.002453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:07.004073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:07.025446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:07.111628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b5464-0716-4698-ba7c-66b3c7854b54 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:07.186910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:07.187515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:07.187792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:07.188059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:07.189659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:07.262692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:07.301468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:07.302937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c87321-898c-437e-a618-dc64cc693ad7 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:07.321338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 672/1323] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:09.296781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657602-5041-4bba-b94e-3b4c22fc4725 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:09.297307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657602-5041-4bba-b94e-3b4c22fc4725 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:09.297307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4657602-5041-4bba-b94e-3b4c22fc4725 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:09.298600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 652/1324] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:09.303784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3705a5-e04a-4fd3-b435-f2691da25a0f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:09.304690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3705a5-e04a-4fd3-b435-f2691da25a0f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:09.304866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3705a5-e04a-4fd3-b435-f2691da25a0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:09.371334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3b3705a5-e04a-4fd3-b435-f2691da25a0f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:09.372521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3b3705a5-e04a-4fd3-b435-f2691da25a0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:09.470181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 673/1325] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:09.765393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:09.765807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:09.766141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:09.767592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 653/1326] 10.222.0.46 () {54 vars in 965 bytes} [Tue Aug 30 04:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:09.774638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79af5-75ad-4446-a883-0f3320ba1582 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:09.774926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79af5-75ad-4446-a883-0f3320ba1582 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:09.775205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da79af5-75ad-4446-a883-0f3320ba1582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:09.812005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:09.857565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5da79af5-75ad-4446-a883-0f3320ba1582 None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:09.858809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5da79af5-75ad-4446-a883-0f3320ba1582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:09.944095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 674/1327] 10.222.0.46 () {58 vars in 1062 bytes} [Tue Aug 30 04:11:09 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:09.953471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:09.955106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:10.060421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2732fb24-fe31-4356-8e16-e8107d5d768b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:10.134923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:10.135519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:10.135793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:10.136057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:10.137639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:10.154350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:10.250771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:10.252150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa5b358-3cb7-48cf-b501-ef23dc109b6b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:10.271014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 654/1328] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:12.470557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:12.471052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:12.471286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:12.472707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 675/1329] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:12.478807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0f43-0d41-499e-a1a5-98b3da8df4ae None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:12.479237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0f43-0d41-499e-a1a5-98b3da8df4ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:12.479333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2e0f43-0d41-499e-a1a5-98b3da8df4ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:12.560952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4c2e0f43-0d41-499e-a1a5-98b3da8df4ae None None] MFA Rules not processed for user `0674c5f80c3443de9ee43f655c58e809`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:12.562189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4c2e0f43-0d41-499e-a1a5-98b3da8df4ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:12.635914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:12.672283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 655/1330] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:12 2022] POST /identity/v3/auth/tokens => generated 3243 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:11:12.967032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:12.968602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:13.072194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee200a20-7e43-4d66-9790-36efc7bef088 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:13.091883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:13.151512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:13.152117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:13.152397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:13.152667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:13.154300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:13.282828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:13.284769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f968282-b5b3-40e4-8242-028df3551395 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:13.311385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 676/1331] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:12 2022] GET /identity/v3/auth/tokens => generated 3243 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:15.685352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:15.685925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:15.685925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:15.687290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 656/1332] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:15.692514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f482-681f-44fe-a6cb-79c8d7c3994d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:15.692901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f482-681f-44fe-a6cb-79c8d7c3994d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:15.693093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f909f482-681f-44fe-a6cb-79c8d7c3994d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:15.758428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:15.761455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f909f482-681f-44fe-a6cb-79c8d7c3994d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:15.763171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f909f482-681f-44fe-a6cb-79c8d7c3994d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:15.859129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 677/1333] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:16.153040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:16.154627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:16.259929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8aa696-4e85-424c-bab3-4bf3e23a63a2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:16.334226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:16.335129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:16.335622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:16.335926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:16.337644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:16.369489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:16.455949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:16.457289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b69a706-b7da-4954-a735-5d5e652adc18 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:16.475357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 657/1334] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:18.462055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:18.462731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:18.462731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:18.463993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 678/1335] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:18.469144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f9326-fa99-4462-800a-d701c3ee5ae4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:18.469587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f9326-fa99-4462-800a-d701c3ee5ae4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:18.469695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0f9326-fa99-4462-800a-d701c3ee5ae4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:18.535275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1f0f9326-fa99-4462-800a-d701c3ee5ae4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:18.536470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1f0f9326-fa99-4462-800a-d701c3ee5ae4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:18.608812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:18.631520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 658/1336] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:18.953185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:18.954800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:19.060519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80a4e4c2-ddf3-4853-a86f-a93c93ad768f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:19.095311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:19.140163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:19.140782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:19.141065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:19.141327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:19.142947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:19.252343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:19.253767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a125a6-e8a6-480b-b7b4-239055d9799f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:19.271446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 679/1337] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:21.772673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:21.773780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:21.773914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:21.775470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 659/1338] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:21.780651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4939-f35c-4ac1-aec6-26db8ae43717 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:21.781083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4939-f35c-4ac1-aec6-26db8ae43717 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:21.781194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7e4939-f35c-4ac1-aec6-26db8ae43717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:21.848266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fa7e4939-f35c-4ac1-aec6-26db8ae43717 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:21.849469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fa7e4939-f35c-4ac1-aec6-26db8ae43717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:21.857891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:21.948058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 680/1339] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:22.228637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:22.230169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:22.337817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d141c32-6b2f-45d0-97ce-b579389d6f8b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:22.411001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:22.411601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:22.411874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:22.412136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:22.414289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:22.448880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:22.528277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:22.529633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093adf9b-27c3-412e-950f-4617b8ee06ba service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:22.548533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 660/1340] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:22 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:26.436747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:26.437268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:26.437268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:26.438698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 681/1341] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:26.444084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344d46-5187-4c2e-ba63-ee6ea489cbb4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:26.444509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344d46-5187-4c2e-ba63-ee6ea489cbb4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:26.444600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7344d46-5187-4c2e-ba63-ee6ea489cbb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:26.511164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a7344d46-5187-4c2e-ba63-ee6ea489cbb4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:26.512377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a7344d46-5187-4c2e-ba63-ee6ea489cbb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:26.602827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:26.614635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 661/1342] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:26.897600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:26.899691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:27.002731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e884243d-99ac-47ab-8974-2063d24932e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:27.042343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:27.078785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:27.079299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:27.079569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:27.079834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:27.081416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:27.194648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:27.196405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf630c51-956f-4760-8230-69f19bebd9a8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:27.214336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 682/1343] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:26 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:29.821176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:29.821736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:29.821736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:29.823252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 662/1344] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:29.829309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5759bb-3e51-42cc-aa79-da6faf2dca74 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:29.829839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5759bb-3e51-42cc-aa79-da6faf2dca74 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:29.829839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5759bb-3e51-42cc-aa79-da6faf2dca74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:29.899217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8c5759bb-3e51-42cc-aa79-da6faf2dca74 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:29.900438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c5759bb-3e51-42cc-aa79-da6faf2dca74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:29.919752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:30.002476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 683/1345] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:29 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:30.287455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:30.289643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:30.395108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78bbec5-6bf4-498a-a240-e016834b3073 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:30.477263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:30.477878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:30.478152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:30.478446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:30.480130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:30.525558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:30.598612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:30.600051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faa64f8a-1203-458f-8967-3f43b5be78b2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:30.619577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 663/1346] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:30 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:34.652713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:34.653309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:34.653309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:34.654653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 684/1347] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:34.659957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a84c0-306b-438b-b538-e3804797d9a3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:34.660381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a84c0-306b-438b-b538-e3804797d9a3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:34.660520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-238a84c0-306b-438b-b538-e3804797d9a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:34.745776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-238a84c0-306b-438b-b538-e3804797d9a3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:34.747028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-238a84c0-306b-438b-b538-e3804797d9a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:34.856912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:34.859801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 664/1348] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:35.140595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:35.142138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:35.258120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ce5fb7-9e3f-4072-ae64-05bb82d092b5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:35.310194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:35.342644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:35.343634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:35.343634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:35.343830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:35.345394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:35.470808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:35.472247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b994b3-b003-4aca-a2a9-6d3387117ae8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:35.492179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 685/1349] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:35 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:40.398599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e780c-ffaa-43da-8211-f1971dcc0db2 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:40.400084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e780c-ffaa-43da-8211-f1971dcc0db2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:40.400084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e780c-ffaa-43da-8211-f1971dcc0db2 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:40.401651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 665/1350] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:11:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:40.407601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36d272-bb05-45a7-8a4b-fc812f54aee5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:40.408079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36d272-bb05-45a7-8a4b-fc812f54aee5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:40.408170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c36d272-bb05-45a7-8a4b-fc812f54aee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:40.492777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4c36d272-bb05-45a7-8a4b-fc812f54aee5 None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:40.492976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4c36d272-bb05-45a7-8a4b-fc812f54aee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:40.520056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:40.650514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 686/1351] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:11:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:44.889820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f0bc-5745-468b-a6ea-2f4c4e8f7dc1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:44.891060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f0bc-5745-468b-a6ea-2f4c4e8f7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:44.891440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b3f0bc-5745-468b-a6ea-2f4c4e8f7dc1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:44.892900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 666/1352] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:11:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:44.899109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff7c56-8e59-4da2-8e90-c9c112c6448b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:44.899726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff7c56-8e59-4da2-8e90-c9c112c6448b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:44.899726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cff7c56-8e59-4da2-8e90-c9c112c6448b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:44.979212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9cff7c56-8e59-4da2-8e90-c9c112c6448b None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:44.980413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9cff7c56-8e59-4da2-8e90-c9c112c6448b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:45.055474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:45.095649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 687/1353] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:11:44 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:46.635002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33ce96-9d47-42d3-bb66-4b1b93e15adf None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:46.635567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33ce96-9d47-42d3-bb66-4b1b93e15adf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:46.635567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe33ce96-9d47-42d3-bb66-4b1b93e15adf None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:46.636869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 667/1354] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:46.642036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f120-044c-4ac6-a588-968963292947 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:46.642528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f120-044c-4ac6-a588-968963292947 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:46.642730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd6f120-044c-4ac6-a588-968963292947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:46.723625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0dd6f120-044c-4ac6-a588-968963292947 None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:46.724844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0dd6f120-044c-4ac6-a588-968963292947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:46.838614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 688/1355] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:11:46 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:50.228769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ddb5-6e0f-4b38-99c0-68bfaf5f1ea4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:50.229302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ddb5-6e0f-4b38-99c0-68bfaf5f1ea4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:50.229302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7870ddb5-6e0f-4b38-99c0-68bfaf5f1ea4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:50.230704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 668/1356] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:50.239486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec659a-a057-4038-9161-d53617e391ab None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:50.239943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec659a-a057-4038-9161-d53617e391ab None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:50.242275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ec659a-a057-4038-9161-d53617e391ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:50.293830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:50.346938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b1ec659a-a057-4038-9161-d53617e391ab None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:50.348165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1ec659a-a057-4038-9161-d53617e391ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:50.470376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 689/1357] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:50.779009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f691-8091-4898-96e4-e2d945150801 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:50.779523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f691-8091-4898-96e4-e2d945150801 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:50.779947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e416f691-8091-4898-96e4-e2d945150801 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:50.781688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 669/1358] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 30 04:11:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:50.789644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59426-27a6-46fb-93b8-58055490063e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:50.790240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59426-27a6-46fb-93b8-58055490063e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:50.790688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d59426-27a6-46fb-93b8-58055490063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:50.872233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c5d59426-27a6-46fb-93b8-58055490063e None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:50.873440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c5d59426-27a6-46fb-93b8-58055490063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:50.924873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:50.997770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 690/1359] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 30 04:11:50 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:11:51.005907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416f691-8091-4898-96e4-e2d945150801 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:51.007536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e416f691-8091-4898-96e4-e2d945150801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:51.162645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e416f691-8091-4898-96e4-e2d945150801 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:51.252052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:51.294901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:51.295591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:51.295971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:51.296342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:51.298059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:51.389812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:51.391647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68369b5f-d2a9-450a-b743-6938375b567a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:51.412796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 670/1360] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 407 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:11:53.604996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:53.605533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:53.605533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:53.607885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 691/1361] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:53.612120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542d5c-ca01-4431-93c7-f55b7e4906a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:53.612552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542d5c-ca01-4431-93c7-f55b7e4906a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:53.612664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32542d5c-ca01-4431-93c7-f55b7e4906a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:53.692012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-32542d5c-ca01-4431-93c7-f55b7e4906a6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:53.693307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32542d5c-ca01-4431-93c7-f55b7e4906a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:53.740319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:53.827999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 671/1362] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:54.122218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:54.123870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:54.300847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a58b5d4-10d3-4f0e-a5a9-3a9096204c77 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:54.312798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:54.445963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:54.447175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:54.447609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:54.448124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:54.450506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:54.568055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:54.570791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737f4ec2-15cc-4b6f-b464-da73632beddf service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:54.599203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 692/1363] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:56.895280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca0397-901f-4eaf-b82c-f716483e0ede None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:56.895847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca0397-901f-4eaf-b82c-f716483e0ede None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:56.895847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ca0397-901f-4eaf-b82c-f716483e0ede None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:56.897158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 672/1364] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:56.902914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85922e-4d4b-4740-9a65-effa61dcb363 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:56.903355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85922e-4d4b-4740-9a65-effa61dcb363 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:56.903451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b85922e-4d4b-4740-9a65-effa61dcb363 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:56.919630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:56.974529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3b85922e-4d4b-4740-9a65-effa61dcb363 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:56.975725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3b85922e-4d4b-4740-9a65-effa61dcb363 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:57.079528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 693/1365] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:57.370624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:57.370903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:57.371380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:57.372720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 673/1366] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 30 04:11:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:57.378242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f9db2-7832-429d-9a3b-46c120d47dae None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:57.378718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f9db2-7832-429d-9a3b-46c120d47dae None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:57.379063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f9db2-7832-429d-9a3b-46c120d47dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:57.459639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-353f9db2-7832-429d-9a3b-46c120d47dae None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:57.460917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-353f9db2-7832-429d-9a3b-46c120d47dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:57.496250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:57.591455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 694/1367] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 30 04:11:57 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:57.600424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:57.602977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:57.771730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b743b66-6090-4466-82f2-d7c8915985b9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:57.794161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:57.908697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:57.909410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:57.909673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:57.909848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:57.911448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:58.011006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:58.012703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57dae352-29de-4c4f-b791-afdfcbbcc22e service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:58.032184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 674/1368] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:11:58.401253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369f6c-716d-4fde-956c-0731bf2b826a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:58.401788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369f6c-716d-4fde-956c-0731bf2b826a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:58.401788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56369f6c-716d-4fde-956c-0731bf2b826a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:58.404391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 695/1369] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:11:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:58.408829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00015266-63c3-4bc7-930c-89ea79488cf1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:58.409219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00015266-63c3-4bc7-930c-89ea79488cf1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:58.409590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00015266-63c3-4bc7-930c-89ea79488cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:58.449378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:58.488992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-00015266-63c3-4bc7-930c-89ea79488cf1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:58.490510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-00015266-63c3-4bc7-930c-89ea79488cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:58.610283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 675/1370] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:11:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:59.200811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:59.200811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:59.200811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:59.203437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 696/1371] 10.222.0.46 () {54 vars in 964 bytes} [Tue Aug 30 04:11:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:11:59.207790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cea1-02b9-4348-ba40-d9a8a11c531a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:59.207790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cea1-02b9-4348-ba40-d9a8a11c531a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:59.207790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c264cea1-02b9-4348-ba40-d9a8a11c531a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:59.293936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c264cea1-02b9-4348-ba40-d9a8a11c531a None None] MFA Rules not processed for user `cefcf250ee17414389d3014e8755416e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:11:59.295214 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c264cea1-02b9-4348-ba40-d9a8a11c531a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:59.344369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:59.382918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 676/1372] 10.222.0.46 () {58 vars in 1061 bytes} [Tue Aug 30 04:11:59 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:11:59.387526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:11:59.389155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:59.501413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6884b9fc-15c1-4961-82ef-a0cc4b1b96f0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:11:59.546561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:11:59.585063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:11:59.585956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:11:59.586354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:11:59.586822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:11:59.588472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:11:59.732246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:11:59.734201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff66173d-7929-4436-8f3b-2276a92be6bb service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:11:59.762389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 697/1373] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:11:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:00.044871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:00.045731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:00.046819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:00.049694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 677/1374] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:00 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:00.055351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48117839-1fc2-4d89-a44b-af74d6e206c9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:00.055659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48117839-1fc2-4d89-a44b-af74d6e206c9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:00.055928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48117839-1fc2-4d89-a44b-af74d6e206c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:00.128514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-48117839-1fc2-4d89-a44b-af74d6e206c9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:00.129718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-48117839-1fc2-4d89-a44b-af74d6e206c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:00.150292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:00.227626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 698/1375] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:00 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:00.532521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:00.534069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:00.686863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103c62e4-2616-40bc-92bf-cfe728629ccc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:00.743192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:00.807416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:00.808218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:00.808600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:00.808974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:00.810755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:00.902511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:00.904096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0a48c3-58d7-4daf-8a22-5601d56bf0bb service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:00.922754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 678/1376] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:00 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:02.969552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:02.970146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:02.970146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:02.971688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 699/1377] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:02.976977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019afff5-5990-4fdb-861d-6e7919ff0745 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:02.977407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019afff5-5990-4fdb-861d-6e7919ff0745 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:02.977505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-019afff5-5990-4fdb-861d-6e7919ff0745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:03.044839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-019afff5-5990-4fdb-861d-6e7919ff0745 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:03.046052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-019afff5-5990-4fdb-861d-6e7919ff0745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:03.052740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:03.150481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 679/1378] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:03.467385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:03.469188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:03.637693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7863608a-47a9-4fc6-aea7-f993712a8059 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:03.641811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:03.765211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:03.765725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:03.766000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:03.766278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:03.767900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:03.884605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:03.886179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff86d1ae-3375-49a6-bd31-d960497157d0 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:03.906340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 700/1379] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:05.910846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:05.911436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:05.911436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:05.912865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 680/1380] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:12:05.918642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7cb2-7132-408a-bdfd-9692aace05fc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:05.919126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7cb2-7132-408a-bdfd-9692aace05fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:05.919224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da7cb2-7132-408a-bdfd-9692aace05fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:05.930620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:05.988440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f2da7cb2-7132-408a-bdfd-9692aace05fc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:05.989914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f2da7cb2-7132-408a-bdfd-9692aace05fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:06.097214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 701/1381] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:06.386934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:06.388565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:06.546277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:06.563022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c0067a-1be3-47e1-b530-c36af10778b5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:06.690527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:06.691052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:06.691328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:06.691597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:06.693195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:06.794627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:06.796235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23292203-4404-4769-bf99-bd5a67c02aeb service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:06.821120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:06.826437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 681/1382] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:08.845798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:08.846380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:08.846380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:08.847870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 702/1383] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:08.853090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db61b04-c9aa-42bf-8154-cd88b66574c4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:08.853601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db61b04-c9aa-42bf-8154-cd88b66574c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:08.853601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db61b04-c9aa-42bf-8154-cd88b66574c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:08.924586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2db61b04-c9aa-42bf-8154-cd88b66574c4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:08.925817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2db61b04-c9aa-42bf-8154-cd88b66574c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:09.026886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 682/1384] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:09.404956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:09.406589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:09.501159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:09.566679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cb6e12-cffd-42c2-b33a-98cb311ebabc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:09.694146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:09.695580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:09.695954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:09.696307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:09.698500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:09.758627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:09.807525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:09.809216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31828ed8-f456-4b64-bb58-d4e76cb94a97 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:09.828778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 703/1385] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:11.798586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:11.799264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:11.799264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:11.800490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 683/1386] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:11.806994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedefc8-20b3-4fd2-b511-d7744a68c049 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:11.807926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedefc8-20b3-4fd2-b511-d7744a68c049 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:11.808287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcedefc8-20b3-4fd2-b511-d7744a68c049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:11.887207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dcedefc8-20b3-4fd2-b511-d7744a68c049 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:11.888949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dcedefc8-20b3-4fd2-b511-d7744a68c049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:11.999112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 704/1387] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:12.293305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:12.294918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:12.380883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:12.457990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803963e3-7b6e-4f9f-b7f8-503fe2b6ca5f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:12.590972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:12.592023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:12.592419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:12.592709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:12.594636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:12.636841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:12.690134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:12.691749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89580b2-5ff5-423f-bf5c-a4bdb2523922 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:12.710905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 684/1388] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:14.750217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae00d-ca7d-44cf-91bb-fc4b77706edf None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:14.750771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae00d-ca7d-44cf-91bb-fc4b77706edf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:14.750771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2eae00d-ca7d-44cf-91bb-fc4b77706edf None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:14.752109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 705/1389] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:14.757679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108408-cef0-4273-a9f3-8ef2a35d11bb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:14.758346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108408-cef0-4273-a9f3-8ef2a35d11bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:14.758346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea108408-cef0-4273-a9f3-8ef2a35d11bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:14.832389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ea108408-cef0-4273-a9f3-8ef2a35d11bb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:14.833606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ea108408-cef0-4273-a9f3-8ef2a35d11bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:14.936307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 685/1390] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:15.074916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.075176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.075347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.076839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 706/1391] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:15.082071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506a03-ebba-4788-b427-52fddf5aa741 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.082511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506a03-ebba-4788-b427-52fddf5aa741 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.082786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11506a03-ebba-4788-b427-52fddf5aa741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.100704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:15.156493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-11506a03-ebba-4788-b427-52fddf5aa741 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:15.157717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-11506a03-ebba-4788-b427-52fddf5aa741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:15.225914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:15.228366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:15.262507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:15.267481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 686/1392] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:15.377980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a3459-1bce-4c88-92f0-6604c367e1cd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:15.480198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:15.493256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:15.493852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.494131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.494426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.496014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:15.576377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ff44-0935-45df-bfe0-a1d3c409f5da None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.576661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ff44-0935-45df-bfe0-a1d3c409f5da None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.576936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b0ff44-0935-45df-bfe0-a1d3c409f5da None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.578318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 687/1393] 10.222.0.46 () {54 vars in 967 bytes} [Tue Aug 30 04:12:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:12:15.583145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e0e05-0fd2-48ce-be9c-14cf0cb03e3a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.583427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e0e05-0fd2-48ce-be9c-14cf0cb03e3a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.583698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831e0e05-0fd2-48ce-be9c-14cf0cb03e3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.588270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:15.589996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:15.609630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 707/1394] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:12:15.666594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-831e0e05-0fd2-48ce-be9c-14cf0cb03e3a None None] MFA Rules not processed for user `288a7a6690d3480db235d2f594bc0acd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:15.667941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-831e0e05-0fd2-48ce-be9c-14cf0cb03e3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:15.709376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:15.761266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 688/1395] 10.222.0.46 () {58 vars in 1064 bytes} [Tue Aug 30 04:12:15 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:12:15.769079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:15.770362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:15.853711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d5f74f4-144d-4736-a827-dde2880d63b9 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:15.895539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:15.930436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:15.931036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:15.931311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:15.931576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:15.933219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:16.045954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:16.052867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66055da2-1439-4864-b0d2-2a905cccd2fc service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:16.077709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 708/1396] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:16.849561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23628-422d-4fc0-aec4-e20811d3317f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:16.850061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23628-422d-4fc0-aec4-e20811d3317f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:16.850343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e23628-422d-4fc0-aec4-e20811d3317f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:16.918837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-87e23628-422d-4fc0-aec4-e20811d3317f None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:16.920057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87e23628-422d-4fc0-aec4-e20811d3317f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:17.006484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 689/1397] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:12:16 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:17.057254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:17.057761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:17.057857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:17.059214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 709/1398] 10.222.0.46 () {54 vars in 943 bytes} [Tue Aug 30 04:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:17.065626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e234-196a-4a14-ac3a-0837af4d6d85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:17.066011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e234-196a-4a14-ac3a-0837af4d6d85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:17.066401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec4e234-196a-4a14-ac3a-0837af4d6d85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:17.137041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:17.156428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4ec4e234-196a-4a14-ac3a-0837af4d6d85 None None] MFA Rules not processed for user `fb30d87ff47b49e39091ef401f10ff59`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:17.157627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4ec4e234-196a-4a14-ac3a-0837af4d6d85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:17.263178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 690/1399] 10.222.0.46 () {58 vars in 1040 bytes} [Tue Aug 30 04:12:17 2022] POST /identity/v3/auth/tokens => generated 3304 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:17.272188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:17.274154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:17.351793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:17.413958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14759e50-533e-4d08-b217-b58de41e37bb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:17.519537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:17.520139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:17.520420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:17.520685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:17.522339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:17.582113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:17.588169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63ead-44be-461f-a1a7-334f30a46845 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:17.588453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63ead-44be-461f-a1a7-334f30a46845 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:17.588728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea63ead-44be-461f-a1a7-334f30a46845 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:17.593815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 691/1400] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:12:17.595076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097178-d456-4e6a-b8aa-b2bf655f8da1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:17.595357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097178-d456-4e6a-b8aa-b2bf655f8da1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:17.595628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a097178-d456-4e6a-b8aa-b2bf655f8da1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:17.609723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:17.611364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:17.620966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 710/1401] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:12:17 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:12:17.668668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5a097178-d456-4e6a-b8aa-b2bf655f8da1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:17.669978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5a097178-d456-4e6a-b8aa-b2bf655f8da1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:17.732927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:17.781251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 692/1402] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:12:18.077556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:18.078895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:18.173638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378ddb4f-6a22-45b9-a19c-2cc40134b636 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:18.285491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:18.298090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:18.298932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:18.299209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:18.299476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:18.301071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:18.406688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:18.408358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4a15d79-eb9a-4a75-8d5b-c01ce220067f service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:18.430891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 711/1403] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:20.440656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:20.441195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:20.441195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:20.442610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 693/1404] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:20.448095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2a14-2b1c-4c8c-8bca-9ddef3467a50 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:20.448915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2a14-2b1c-4c8c-8bca-9ddef3467a50 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:20.449401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90d2a14-2b1c-4c8c-8bca-9ddef3467a50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:20.529001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e90d2a14-2b1c-4c8c-8bca-9ddef3467a50 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:20.530384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e90d2a14-2b1c-4c8c-8bca-9ddef3467a50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:20.603734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:20.715537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 712/1405] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:21.006207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:21.008995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:21.184924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e3a3f7-7636-4a39-90c5-1276fbfefdd8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:21.232371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:21.324761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:21.325371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:21.325645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:21.325937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:21.327579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:21.428520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:21.430843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a291d6-606b-4c80-b107-0b9566b7b1e3 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:21.456024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 694/1406] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:23.544613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:23.545141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:23.545141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:23.546460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 713/1407] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:12:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:23.551706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e15f41-f46d-42e3-89c8-eb6994117f01 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:23.552177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e15f41-f46d-42e3-89c8-eb6994117f01 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:23.552350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e15f41-f46d-42e3-89c8-eb6994117f01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:23.615484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:23.624667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e6e15f41-f46d-42e3-89c8-eb6994117f01 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:23.625954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6e15f41-f46d-42e3-89c8-eb6994117f01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:23.732474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 695/1408] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:12:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:24.030025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:24.031646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:24.213139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e80866cd-fabe-43a0-a3cf-41f8ed99f043 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:24.264157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:24.340440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:24.341049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:24.341328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:24.341597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:24.343244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:24.447953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:24.449556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e514b77b-14fd-4239-ab3a-49ac7b185061 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:24.468887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 714/1409] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:33.730505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=44 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:34.466566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=77 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:40.255202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021cb29-732b-42c9-a697-141ed506c280 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:40.255774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021cb29-732b-42c9-a697-141ed506c280 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:40.255774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8021cb29-732b-42c9-a697-141ed506c280 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:40.257928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 696/1410] 10.222.0.46 () {54 vars in 918 bytes} [Tue Aug 30 04:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:40.263097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aad31b-25a6-43f1-8a6a-e9f09b8289d4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:40.263514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aad31b-25a6-43f1-8a6a-e9f09b8289d4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:40.263616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83aad31b-25a6-43f1-8a6a-e9f09b8289d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:40.339367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-83aad31b-25a6-43f1-8a6a-e9f09b8289d4 None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:40.340604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-83aad31b-25a6-43f1-8a6a-e9f09b8289d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:40.449427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 715/1411] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Aug 30 04:12:40 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:40.463750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d72-2cca-4e52-9781-c173ff0e018f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:40.464035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d72-2cca-4e52-9781-c173ff0e018f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:40.464308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0c9d72-2cca-4e52-9781-c173ff0e018f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:40.465670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 697/1412] 10.222.0.46 () {54 vars in 918 bytes} [Tue Aug 30 04:12:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:40.470395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53c7-b08f-4801-b2bc-8e8c83f72b8d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:40.470682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53c7-b08f-4801-b2bc-8e8c83f72b8d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:40.470951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cef53c7-b08f-4801-b2bc-8e8c83f72b8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:40.543268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:40.546457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3cef53c7-b08f-4801-b2bc-8e8c83f72b8d None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:40.548334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3cef53c7-b08f-4801-b2bc-8e8c83f72b8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:40.674213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 716/1413] 10.222.0.46 () {58 vars in 1015 bytes} [Tue Aug 30 04:12:40 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:43.089688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:43.090273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:43.090273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:43.091605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 698/1414] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:12:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:43.096845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ce7f7-0c68-4d14-8e7c-24a4dfdcf214 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:43.097267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ce7f7-0c68-4d14-8e7c-24a4dfdcf214 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:43.097395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053ce7f7-0c68-4d14-8e7c-24a4dfdcf214 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:43.164565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-053ce7f7-0c68-4d14-8e7c-24a4dfdcf214 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:43.165771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-053ce7f7-0c68-4d14-8e7c-24a4dfdcf214 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:43.221539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:43.263513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 717/1415] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:12:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:43.561563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:43.563179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:43.711177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32dc2e6-84e2-42f6-941a-06347350ad12 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:43.785827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:43.829961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:43.830585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:43.830858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:43.831120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:43.832692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:44.038955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:44.040648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e8af48-9f87-4171-b544-ccbca7766967 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:44.058843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 699/1416] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:44.967151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:44.967729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:44.967729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:44.969055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 718/1417] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:44.973874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490ce4d-2767-4447-8bf1-243f615b017e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:44.974336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490ce4d-2767-4447-8bf1-243f615b017e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:44.974453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2490ce4d-2767-4447-8bf1-243f615b017e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:45.051282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2490ce4d-2767-4447-8bf1-243f615b017e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:45.052495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2490ce4d-2767-4447-8bf1-243f615b017e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:45.080438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:45.152331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 700/1418] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:45.308094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:45.309613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:45.454828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0de7edc-d91a-4269-9884-d8aff1a50e91 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:45.494316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:45.566859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:45.567461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:45.567735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:45.567991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:45.569647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:45.659895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:45.661481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288345db-e072-4b1a-9775-bcbc61811b11 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:45.679852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 719/1419] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:46.574039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:46.574622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:46.574622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:46.576006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 701/1420] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:46.580887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5f0a2-e236-48f0-9fe2-ee741d916684 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:46.581286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5f0a2-e236-48f0-9fe2-ee741d916684 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:46.581439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b5f0a2-e236-48f0-9fe2-ee741d916684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:46.636336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:46.660188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f1b5f0a2-e236-48f0-9fe2-ee741d916684 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:46.661407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1b5f0a2-e236-48f0-9fe2-ee741d916684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:46.756609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 720/1421] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:46.915683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:46.917292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:47.061292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd24d038-4c60-4a6a-bd7c-126d2652c083 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:47.073421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:47.183892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:47.184659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:47.185052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:47.185320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:47.186937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:47.276759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:47.278394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9498e2d6-19a9-4109-9243-0a952382f4af service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:47.296368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 702/1422] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:46 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:48.171903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:48.172505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:48.172505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:48.173762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 721/1423] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:48.178835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28fcc-5ce8-433f-b81f-9ba5f5386010 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:48.179269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28fcc-5ce8-433f-b81f-9ba5f5386010 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:48.179877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b28fcc-5ce8-433f-b81f-9ba5f5386010 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:48.200329 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:48.258415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-85b28fcc-5ce8-433f-b81f-9ba5f5386010 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:48.259638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85b28fcc-5ce8-433f-b81f-9ba5f5386010 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:48.355350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 703/1424] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:48.508384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:48.509816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:48.621288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:48.658829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b95aae0e-8358-4b3a-b845-a84c4903b0a9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:48.773979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:48.774515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:48.774789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:48.775051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:48.776635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:48.854871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:48.869246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:48.870876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3950594b-133c-4ff5-89dc-26e8c4796bb8 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:48.888750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 722/1425] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:49.792738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:49.793317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:49.793544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:49.795772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 704/1426] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:49 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:49.800742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a214b-b00a-49d1-8f44-dcbd238266e3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:49.801197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a214b-b00a-49d1-8f44-dcbd238266e3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:49.801294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560a214b-b00a-49d1-8f44-dcbd238266e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:49.881308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-560a214b-b00a-49d1-8f44-dcbd238266e3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:49.883153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-560a214b-b00a-49d1-8f44-dcbd238266e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:49.976883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 723/1427] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:50.136497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:50.138001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:50.223070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:50.286494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47bdc3ed-6986-4a49-9dd9-1383925e242b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:50.400910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:50.401416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:50.401694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:50.401953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:50.403589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:50.457858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:50.506813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:50.508935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d644306b-d7f5-4761-8c8f-5a34e991b44f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:50.528666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 705/1428] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:51.348051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:51.348606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:51.348606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:51.349949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 724/1429] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:51.355558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3e90-0fe9-43a1-b09a-150d694283b5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:51.355996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3e90-0fe9-43a1-b09a-150d694283b5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:51.356103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc3e90-0fe9-43a1-b09a-150d694283b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:51.431385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cecc3e90-0fe9-43a1-b09a-150d694283b5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:51.432599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cecc3e90-0fe9-43a1-b09a-150d694283b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:51.529711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 706/1430] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:51.688291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:51.689844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:51.738173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:51.838204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591cf4e4-dd4a-42cc-80de-41766d56c158 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:51.952147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:51.952753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:51.953033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:51.953303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:51.954944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:51.962589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:52.047174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:52.048755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b695b685-bfcd-48fb-96ff-71698727951e service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:52.067462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 725/1431] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:52.877053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:52.877613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:52.877613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:52.879025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 707/1432] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:52.884719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddec58f-5e66-46f5-b948-3191dbc4eaea None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:52.885168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddec58f-5e66-46f5-b948-3191dbc4eaea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:52.885278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddec58f-5e66-46f5-b948-3191dbc4eaea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:52.962693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8ddec58f-5e66-46f5-b948-3191dbc4eaea None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:52.963919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ddec58f-5e66-46f5-b948-3191dbc4eaea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:53.033072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:53.063558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 726/1433] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:53.218614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:53.220152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:53.237899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:53.367176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9371e7fd-0c0f-4858-8c26-b2119ad22d5f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:53.455569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:53.482628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:53.483225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:53.483495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:53.483757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:53.485325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:53.574993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:53.576578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad08b2-186d-4f6e-b95f-7112c2512f3b service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:53.594736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 708/1434] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:54.397240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:54.398350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:54.398350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:54.400253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 727/1435] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:54.405269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ba9a-b21c-4e82-b786-3fde4b63f8e3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:54.405679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ba9a-b21c-4e82-b786-3fde4b63f8e3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:54.405811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9945ba9a-b21c-4e82-b786-3fde4b63f8e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:54.481993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9945ba9a-b21c-4e82-b786-3fde4b63f8e3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:54.483233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9945ba9a-b21c-4e82-b786-3fde4b63f8e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:54.524796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:54.580020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 709/1436] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:54.731740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:54.733194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:54.878140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072a877c-2f96-40a9-a520-c6a72ee2a745 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:54.933774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:54.993116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:54.993631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:54.993904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:54.994170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:54.995795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:55.083035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:55.084625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47b23be5-a385-45df-9af1-75dd3666004f service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:55.103292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 728/1437] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:55.931841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:55.932386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:55.932386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:55.933736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 710/1438] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:12:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:55.938811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751721-2deb-4cfd-badf-2733877e468b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:55.939224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751721-2deb-4cfd-badf-2733877e468b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:55.939365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2751721-2deb-4cfd-badf-2733877e468b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:56.017534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f2751721-2deb-4cfd-badf-2733877e468b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:56.018790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f2751721-2deb-4cfd-badf-2733877e468b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:56.020857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:56.117372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 729/1439] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:12:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:56.270345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:56.271922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:56.421007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4cf2658-a406-41aa-aa68-b33077143fd4 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:56.449190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:56.535706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:56.536301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:56.536578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:56.536844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:56.538595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:56.626524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:56.628109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad4ba362-d695-41e0-8480-6174bfb6a9d5 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:56.646725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 711/1440] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:12:58.470234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:58.470801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:58.470801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:58.472139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 730/1441] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:12:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:12:58.478054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71d212-ae60-4f45-bf7c-4fc57de1fe55 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:58.478517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71d212-ae60-4f45-bf7c-4fc57de1fe55 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:58.478642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b71d212-ae60-4f45-bf7c-4fc57de1fe55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:58.519733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:58.546519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1b71d212-ae60-4f45-bf7c-4fc57de1fe55 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:12:58.547724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b71d212-ae60-4f45-bf7c-4fc57de1fe55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:58.643498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 712/1442] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:12:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:12:58.928596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:12:58.930844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:59.040574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1238d8-1b95-45f8-b668-b874368d75ae None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:12:59.073683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:12:59.114947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:12:59.115555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:12:59.115836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:12:59.116109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:12:59.119293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:12:59.235829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:12:59.237254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0ec879-1fe2-424d-96ec-99e869f5581c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:12:59.255590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 731/1443] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:12:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:01.329267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984283-d0e2-4930-8202-f825beecc721 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:01.329818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984283-d0e2-4930-8202-f825beecc721 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:01.329818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db984283-d0e2-4930-8202-f825beecc721 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:01.331224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 713/1444] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:01.337365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bf2f6-05e5-4c0a-a5e0-c2f917be970f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:01.337807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bf2f6-05e5-4c0a-a5e0-c2f917be970f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:01.337921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25bf2f6-05e5-4c0a-a5e0-c2f917be970f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:01.403883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c25bf2f6-05e5-4c0a-a5e0-c2f917be970f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:01.405102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c25bf2f6-05e5-4c0a-a5e0-c2f917be970f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:01.418734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:01.503605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 732/1445] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:01 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:01.786126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db984283-d0e2-4930-8202-f825beecc721 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:01.787674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-db984283-d0e2-4930-8202-f825beecc721 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:01.890650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db984283-d0e2-4930-8202-f825beecc721 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:01.903994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:01.965209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:01.965811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:01.966088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:01.966389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:01.967998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:02.076641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:02.077986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdb8614-6c7d-4b1e-89a3-e661435932ce service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:02.096629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 714/1446] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:01 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:04.343701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:04.344251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:04.344251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:04.345572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 733/1447] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:04.350738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf368-cd93-43ac-b3fe-eac448f8eb4f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:04.353115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf368-cd93-43ac-b3fe-eac448f8eb4f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:04.353115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fdf368-cd93-43ac-b3fe-eac448f8eb4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:04.400561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:04.420085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66fdf368-cd93-43ac-b3fe-eac448f8eb4f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:04.421396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66fdf368-cd93-43ac-b3fe-eac448f8eb4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:04.519803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 715/1448] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:04.803794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:04.805950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:04.909695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301610b7-5ceb-4d68-8a70-c8b2767b56b3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:04.953613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:04.989091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:04.989702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:04.989983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:04.990272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:04.992374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:05.099797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:05.101570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ef2a25-821b-4849-96d5-faf7841ff50d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:05.120537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 734/1449] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:07.339637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:07.340167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:07.340167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:07.341525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 716/1450] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:07.346827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9520-9bb3-4446-a24f-e722f8d06333 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:07.347255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9520-9bb3-4446-a24f-e722f8d06333 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:07.347374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466a9520-9bb3-4446-a24f-e722f8d06333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:07.412205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-466a9520-9bb3-4446-a24f-e722f8d06333 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:07.413949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-466a9520-9bb3-4446-a24f-e722f8d06333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:07.434910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:07.509882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 735/1451] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:07.799098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:07.800694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:07.906702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a0e56-a024-45cb-87e9-ea5f102559b1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:07.926167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:07.981203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:07.981804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:07.982080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:07.982389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:07.983976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:08.092670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:08.094112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b211dc25-00b9-460b-85a0-c6b7722dc097 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:08.114121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 717/1452] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:10.087215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:10.087771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:10.087771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:10.089115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 736/1453] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:10.094650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2368c-4191-45d2-8019-d64b174d1682 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:10.095060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2368c-4191-45d2-8019-d64b174d1682 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:10.095178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee2368c-4191-45d2-8019-d64b174d1682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:10.158669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:10.161813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6ee2368c-4191-45d2-8019-d64b174d1682 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:10.163812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6ee2368c-4191-45d2-8019-d64b174d1682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:10.271669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 718/1454] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:10.564965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:10.566553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:10.676145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dda335c-28b4-474f-a9d0-1a1215a115a2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:10.726135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:10.750741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:10.751348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:10.751627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:10.751898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:10.753513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:10.865746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:10.867651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5b4213-2395-47b3-b2ab-9a8ade73fbfe service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:10.885779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 737/1455] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:12.863224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:12.863765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:12.863765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:12.865125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 719/1456] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:12.871278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16af4-32de-4922-b8be-302e02ccf001 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:12.871734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16af4-32de-4922-b8be-302e02ccf001 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:12.871839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbe16af4-32de-4922-b8be-302e02ccf001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:12.939502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dbe16af4-32de-4922-b8be-302e02ccf001 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:12.940735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dbe16af4-32de-4922-b8be-302e02ccf001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:12.970592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:13.038925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 738/1457] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:13.327021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:13.328548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:13.431998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925b4ab2-b7cc-4de2-b39e-920f65249739 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:13.461153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:13.506389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:13.506999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:13.507923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:13.508434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:13.510074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:13.620799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:13.622222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aeb8dfb4-d8ac-42cf-95d5-5f57991ba493 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:13.641349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 720/1458] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:13 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:15.715104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:15.715691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:15.715691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:15.717048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 739/1459] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:15.722413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa758d-eb7d-468e-88da-63dde2cd6f32 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:15.722793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa758d-eb7d-468e-88da-63dde2cd6f32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:15.722963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa758d-eb7d-468e-88da-63dde2cd6f32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:15.789208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22fa758d-eb7d-468e-88da-63dde2cd6f32 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:15.790542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22fa758d-eb7d-468e-88da-63dde2cd6f32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:15.799243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:15.889466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 721/1460] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:13:16.171782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:16.173319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:16.280175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cfaee95-bc79-458d-93d9-ec9f941c528a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:16.335906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:16.354631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:16.355232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:16.355509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:16.355776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:16.357372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:16.464754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:16.466089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e650fc-069b-483b-b162-a7e150ca9b26 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:16.484267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 740/1461] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:18.463658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:18.464203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:18.464203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:18.465562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 722/1462] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:18.470908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6526ebb-0fc9-46ed-9c16-49c17ff70485 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:18.471363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6526ebb-0fc9-46ed-9c16-49c17ff70485 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:18.471471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6526ebb-0fc9-46ed-9c16-49c17ff70485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:18.539125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b6526ebb-0fc9-46ed-9c16-49c17ff70485 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:18.541093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b6526ebb-0fc9-46ed-9c16-49c17ff70485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:18.584619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:18.658831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 741/1463] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:18.946680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:18.948428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:19.054213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf70cd9-f00a-4035-a2d7-8008b03a47d9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:19.093061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:19.129189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:19.129792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:19.130070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:19.130377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:19.131982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:19.242399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:19.243832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55db1ed3-7d97-4c4f-ac80-74f5acd54696 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:19.265060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 723/1464] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:21.220102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:21.220646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:21.220646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:21.222006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 742/1465] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:21.228034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8873-52bd-4c2b-9723-3a3d904df25d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:21.228458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8873-52bd-4c2b-9723-3a3d904df25d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:21.228743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52d8873-52bd-4c2b-9723-3a3d904df25d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:21.324094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c52d8873-52bd-4c2b-9723-3a3d904df25d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:21.325332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c52d8873-52bd-4c2b-9723-3a3d904df25d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:21.343628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:21.422565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 724/1466] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:21.710009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:21.711704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:21.817343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3b3fb0-5460-417c-9d1b-93eac2f7ed52 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:21.884780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:21.892060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:21.892674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:21.892960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:21.893260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:21.894924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:22.009690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:22.011064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efabdb2e-5f4d-4258-aa83-edbf0ae94b3f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:22.030396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 743/1467] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:23.994948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27755e-c396-4332-927d-1212396e7245 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:23.995559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27755e-c396-4332-927d-1212396e7245 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:23.995559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27755e-c396-4332-927d-1212396e7245 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:23.996793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 725/1468] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:13:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:24.002001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2cb9f-e463-4ee2-aaf3-c868137e8c65 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:24.002472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2cb9f-e463-4ee2-aaf3-c868137e8c65 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:24.002581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa2cb9f-e463-4ee2-aaf3-c868137e8c65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:24.070207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0aa2cb9f-e463-4ee2-aaf3-c868137e8c65 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:24.071464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0aa2cb9f-e463-4ee2-aaf3-c868137e8c65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:24.119743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:24.169170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 744/1469] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:13:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:24.475581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27755e-c396-4332-927d-1212396e7245 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:24.477763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ee27755e-c396-4332-927d-1212396e7245 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:24.607703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27755e-c396-4332-927d-1212396e7245 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:24.662445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:24.691863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:24.692467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:24.692739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:24.693005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:24.695541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:24.826852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:24.828387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9c184-1e37-405c-8a6c-be5e4c5a710c service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:24.849033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 726/1470] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:34.167348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=22 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:34.846625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=66 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:44.796799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2de6-7b4f-4867-96a6-addc905e226a None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:44.797371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2de6-7b4f-4867-96a6-addc905e226a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:44.797371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35bd2de6-7b4f-4867-96a6-addc905e226a None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:44.798763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 745/1471] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:44.804739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec985-2380-43be-8365-b1552f64cad8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:44.805200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec985-2380-43be-8365-b1552f64cad8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:44.805311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadec985-2380-43be-8365-b1552f64cad8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:44.875268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-aadec985-2380-43be-8365-b1552f64cad8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:44.876473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-aadec985-2380-43be-8365-b1552f64cad8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:44.970980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 727/1472] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:45.275882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:45.276165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:45.276436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:45.277774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 746/1473] 10.222.0.46 () {54 vars in 967 bytes} [Tue Aug 30 04:13:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:45.282493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fabd-4554-471c-8434-90e72cc69d17 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:45.282774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fabd-4554-471c-8434-90e72cc69d17 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:45.283045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f8fabd-4554-471c-8434-90e72cc69d17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:45.359260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d0f8fabd-4554-471c-8434-90e72cc69d17 None None] MFA Rules not processed for user `288a7a6690d3480db235d2f594bc0acd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:45.360462 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d0f8fabd-4554-471c-8434-90e72cc69d17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:45.380443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:45.435918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 728/1474] 10.222.0.46 () {58 vars in 1064 bytes} [Tue Aug 30 04:13:45 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:13:45.443486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:45.445092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:45.548385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff641b3c-e645-461f-ba8c-62eb1bcc0493 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:45.624980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:45.625592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:45.625870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:45.626136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:45.627779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:45.682534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:45.737334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:45.738801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd02fc89-3350-47f7-99b5-424daa6c1bb1 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:45.759091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 747/1475] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:47.767591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:47.768166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:47.768166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:47.769541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 729/1476] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:47.776369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45f95-40f8-434a-bb5c-b95509c9764c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:47.776816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45f95-40f8-434a-bb5c-b95509c9764c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:47.776937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c45f95-40f8-434a-bb5c-b95509c9764c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:47.842730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-77c45f95-40f8-434a-bb5c-b95509c9764c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:47.843954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-77c45f95-40f8-434a-bb5c-b95509c9764c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:47.940271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 748/1477] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:48.223673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:48.225796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:48.355153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee9170f-e695-46f9-b358-b3accc5cb782 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:48.404265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:48.449200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:48.449801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:48.450076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:48.450377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:48.451014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:48.452748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:13:48.452748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:13:48.452748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:13:48.453892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f708c6-e4dc-4b12-a064-b7f0065dad86 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:48.473890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 730/1478] 10.222.0.46 () {56 vars in 1108 bytes} [Tue Aug 30 04:13:48 2022] GET /identity/v3/projects => generated 2553 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:13:50.435993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:50.436556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:50.436556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:50.437875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 749/1479] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:50.443071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2306-8f23-4add-9def-6fda026a8db1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:50.444059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2306-8f23-4add-9def-6fda026a8db1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:50.444220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cf2306-8f23-4add-9def-6fda026a8db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:50.509711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b6cf2306-8f23-4add-9def-6fda026a8db1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:50.510959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b6cf2306-8f23-4add-9def-6fda026a8db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:50.609134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 731/1480] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:50.910882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:50.913079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:50.923016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:51.057217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a43b1-18ea-465c-b220-cf57d13d1f09 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:51.139978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:51.173377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:51.173914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:51.174191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:51.174502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:51.176172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:51.264841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:51.266451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5e22fb-920b-41a5-9382-09e616604ef5 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:51.284875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 750/1481] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:53.403475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:53.404105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:53.404105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:53.405418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 732/1482] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:53.410851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d08f4-fd7b-4511-90fa-731908abe2ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:53.411309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d08f4-fd7b-4511-90fa-731908abe2ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:53.411414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35d08f4-fd7b-4511-90fa-731908abe2ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:53.601161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e35d08f4-fd7b-4511-90fa-731908abe2ac None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:53.602392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e35d08f4-fd7b-4511-90fa-731908abe2ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:53.648506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:53.701478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 751/1483] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:53.999299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:54.000834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:54.026140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:54.106360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df9ee35-cf4f-42b9-a93f-bb773f7cf487 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:54.179619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:54.180215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:54.180487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:54.180755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:54.182384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:54.255398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:54.291729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:54.293195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b0392d-8599-49b9-9355-2e62dc564b12 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:54.312013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 733/1484] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:56.307819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:56.308363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:56.308363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:56.309713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 752/1485] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:56.315087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba0a8-95a7-4640-b5c3-4f4187ba0a26 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:56.315524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba0a8-95a7-4640-b5c3-4f4187ba0a26 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:56.315623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5ba0a8-95a7-4640-b5c3-4f4187ba0a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:56.383825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3e5ba0a8-95a7-4640-b5c3-4f4187ba0a26 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:56.385048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3e5ba0a8-95a7-4640-b5c3-4f4187ba0a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:56.481919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 734/1486] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:56.774504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:56.776062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:56.895425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aee3453-d5a1-4079-8945-53e8657b4723 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:56.967541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:56.969323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:56.969934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:56.970421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:56.970917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:56.972725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:57.080545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:57.081887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20a21502-7708-444f-a5b3-bc132a55753c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:57.100242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 753/1487] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:56 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:13:59.120556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:59.121125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:59.121125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:59.122470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 735/1488] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:13:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:13:59.127789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b42a13-564c-4a39-a019-b1546e6711d4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:59.128238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b42a13-564c-4a39-a019-b1546e6711d4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:59.128389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b42a13-564c-4a39-a019-b1546e6711d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:59.197194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-92b42a13-564c-4a39-a019-b1546e6711d4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:13:59.198438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-92b42a13-564c-4a39-a019-b1546e6711d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:59.251642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:59.295921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 754/1489] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:13:59 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:13:59.587958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:13:59.589494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:59.625471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:59.695139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bab6a5-afa5-45a4-8ade-ddad74093bee None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:13:59.768157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:13:59.768763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:13:59.769041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:13:59.769346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:13:59.771639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:13:59.854602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:13:59.884155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:13:59.885629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded3dc12-13ff-456f-a4e7-fcf0a19135a5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:13:59.905057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 736/1490] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:13:59 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:14:02.009731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:14:02.010325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:14:02.010325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:14:02.011655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 755/1491] 10.222.0.46 () {54 vars in 927 bytes} [Tue Aug 30 04:14:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:14:02.016994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d869c-450e-4b52-95ab-9c3666e19ac9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:14:02.017428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d869c-450e-4b52-95ab-9c3666e19ac9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:14:02.017549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d20d869c-450e-4b52-95ab-9c3666e19ac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:14:02.084881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d20d869c-450e-4b52-95ab-9c3666e19ac9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:14:02.086086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d20d869c-450e-4b52-95ab-9c3666e19ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:14:02.182039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 737/1492] 10.222.0.46 () {58 vars in 1024 bytes} [Tue Aug 30 04:14:02 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:14:02.473289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:14:02.475364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:14:02.579796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c037970b-1808-4ff1-a6a8-a2e9f85f20c2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:14:02.652555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:14:02.653162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:14:02.653438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:14:02.653705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:14:02.655367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:14:02.664437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:14:02.762901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:14:02.764225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-076f1262-a0f6-496f-963d-cd66fc7c102b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:14:02.782002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 756/1493] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:14:02 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:14:12.180227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=82 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:14:12.779928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=50 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:17.191106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:17.191711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:17.191711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:17.266947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:15:17.268793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:17.365184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 738/1494] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:15:17.373362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:17.373842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:17.373952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:17.455731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:15:17.456875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:17.557772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 757/1495] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:15:17 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:15:17.562597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:17.563833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:17.576326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd5bfa8-679d-4400-b1c5-2ccd8f00cefd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:17.640981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:17.677084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:17.677689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:17.677965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:17.678229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:17.678903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:15:17.679202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:17.680217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:17.680217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:15:17.680217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:15:17.681096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:17.692872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 739/1496] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:15:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:15:17.701531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:17.710903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3973bca6-043e-4701-8e6f-893b2b0c3a3e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:17.774452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:17.809069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:17.809688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:17.809964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:17.810239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:17.810905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:17.812668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:17.812668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:15:17.812668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:15:17.813972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:17.877492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 758/1497] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:15:17 2022] POST /identity/v3/projects => generated 377 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:15:17.881678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:17.883264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:17.919050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ef3476-c34a-415a-9c5d-d12064726904 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:18.013890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:18.014427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.014702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.014968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.015580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:18.016581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:18.016581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:15:18.016581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:15:18.018908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:18.062727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 740/1498] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:15:17 2022] POST /identity/v3/users => generated 289 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:15:18.067505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:18.069057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.132621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17012c4b-5b0a-4ae7-90eb-cadfd56395f4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:18.219380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:18.230601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:18.231203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.231481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.231745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.232378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:18.233387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:18.233387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:15:18.233387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:15:18.234277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:18.246663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 759/1499] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:15:18.250747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:18.253079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.264602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504a80cb-a2d2-43e9-8a55-2ca7fc219a45 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:18.290663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:18.362047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:18.362681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.362962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.363228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.363863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:18.364881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:18.364881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:15:18.364881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:15:18.365760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11eb3a5e-6777-4e55-af2f-fafd9cb1f76d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:18.378803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 741/1500] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:15:18 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:15:18.383005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:18.384539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.417703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb26e0d-c9ef-4b2d-b790-a528756ab590 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:18.513203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:18.513851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.514129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.514426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] PATH_INFO: `/v3/projects/7e460187fa974326b9b3704b0595e370/users/c12d25ca92fe46da8c9f6bda0024ee2b/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.536325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e460187fa974326b9b3704b0595e370, user_id=c12d25ca92fe46da8c9f6bda0024ee2b, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:18.537278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:18.537278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:15:18.537278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:15:18.539226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc573c8-269b-466e-bcf3-b2d0b59b776d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:18.565234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 760/1501] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:15:18 2022] PUT /identity/v3/projects/7e460187fa974326b9b3704b0595e370/users/c12d25ca92fe46da8c9f6bda0024ee2b/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:15:18.577468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32a2c-53d9-4b3e-870c-c1f44f0714e8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.577775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32a2c-53d9-4b3e-870c-c1f44f0714e8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.578074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d32a2c-53d9-4b3e-870c-c1f44f0714e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.637886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d2d32a2c-53d9-4b3e-870c-c1f44f0714e8 None None] MFA Rules not processed for user `c12d25ca92fe46da8c9f6bda0024ee2b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:15:18.639122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d2d32a2c-53d9-4b3e-870c-c1f44f0714e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.716874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 742/1502] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:15:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:15:18.730109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.730422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.730694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.732009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 761/1503] 10.222.0.46 () {54 vars in 964 bytes} [Tue Aug 30 04:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:15:18.736974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cc124-b572-46a2-afd1-73f647badb84 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:18.737263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cc124-b572-46a2-afd1-73f647badb84 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:18.737590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19cc124-b572-46a2-afd1-73f647badb84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:18.771262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:18.816501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a19cc124-b572-46a2-afd1-73f647badb84 None None] MFA Rules not processed for user `cefcf250ee17414389d3014e8755416e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:15:18.817597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a19cc124-b572-46a2-afd1-73f647badb84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.894344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 743/1504] 10.222.0.46 () {58 vars in 1061 bytes} [Tue Aug 30 04:15:18 2022] POST /identity/v3/auth/tokens => generated 3120 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:15:18.902423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:18.903925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:18.953759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:19.008421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7784dd76-5e5a-406e-ad73-423c7ff5f3af None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:19.081474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:19.082081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:19.082393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:19.082666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:19.084263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:19.181440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:19.182893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:19.186356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:19.202113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 762/1505] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:15:18 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:15:19.247666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04300d-e1c3-452a-8c58-9b8a440a8039 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:19.247950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04300d-e1c3-452a-8c58-9b8a440a8039 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:19.248223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e04300d-e1c3-452a-8c58-9b8a440a8039 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:19.249467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 744/1506] 10.222.0.46 () {52 vars in 830 bytes} [Tue Aug 30 04:15:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:15:19.254125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:19.255365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:19.355527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3fbbc76-f0ad-45ee-87a0-86f466ab006c service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:19.450788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:19.451470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:19.451673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:19.451936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] PATH_INFO: `/v3/users/c12d25ca92fe46da8c9f6bda0024ee2b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:19.464099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c12d25ca92fe46da8c9f6bda0024ee2b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:19.465125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:19.465125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:15:19.465125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:15:19.466351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:19.476493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:19.535429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c12d25ca92fe46da8c9f6bda0024ee2b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:19.557389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c12d25ca92fe46da8c9f6bda0024ee2b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:19.557889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c12d25ca92fe46da8c9f6bda0024ee2b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:19.558710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c12d25ca92fe46da8c9f6bda0024ee2b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:19.568842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c12d25ca92fe46da8c9f6bda0024ee2b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:19.589280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 763/1507] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:15:19 2022] DELETE /identity/v3/users/c12d25ca92fe46da8c9f6bda0024ee2b => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:15:19.593832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:19.594339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:19.594438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:19.651922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:15:19.653047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:19.730844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:19.745005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 745/1508] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:15:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:15:19.754690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:19.755915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:19.860402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bfa470-1a78-40a7-8e2c-2e8c00639814 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:19.917066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:19.936729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:19.937344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:19.937621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:19.937889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:19.939527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:20.048030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:20.050034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacb6257-fa3a-44e2-90cd-5e7a22f729b2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:20.071289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 764/1509] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:15:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:15:20.442664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:15:20.443890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:15:20.455816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf5308e-02f9-42ee-bcfc-8310c4e2c49a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:15:20.562600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:15:20.563224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:15:20.563495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:15:20.563763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] PATH_INFO: `/v3/projects/7e460187fa974326b9b3704b0595e370` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:15:20.574409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e460187fa974326b9b3704b0595e370)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:15:20.575429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:15:20.575429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:15:20.575429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:15:20.576607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:15:20.636779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e460187fa974326b9b3704b0595e370', 'request_id': 'req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:20.646636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e460187fa974326b9b3704b0595e370', 'request_id': 'req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:20.660496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e460187fa974326b9b3704b0595e370', 'request_id': 'req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:20.683303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] The token cache is being invalidate because project 7e460187fa974326b9b3704b0595e370 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:15:20.683590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:20.684116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0faa719d-8fc2-4527-bb1f-78b77a3eeddb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:15:20.685519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 746/1510] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:15:20 2022] DELETE /identity/v3/projects/7e460187fa974326b9b3704b0595e370 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:15:30.068889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=62 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:15:30.681164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:16:53.977951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:53.978562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:53.978562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:53.979821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 765/1511] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:16:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:16:53.985408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7d418-5537-4424-bc20-179868be49c4 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:53.985832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7d418-5537-4424-bc20-179868be49c4 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:53.985943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc7d418-5537-4424-bc20-179868be49c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:54.067890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6dc7d418-5537-4424-bc20-179868be49c4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:16:54.069113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6dc7d418-5537-4424-bc20-179868be49c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:16:54.178459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 747/1512] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:16:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:16:54.333880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:16:54.336008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:16:54.461666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea879d0-c83d-48b7-b1b3-5dd520bde1db None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:16:54.468144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:16:54.468784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:54.469064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:54.469337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:54.477406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:16:54.479019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb11af9-7cb6-47a8-a27b-871200a6e586 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:16:54.489234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 766/1513] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:16:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:16:55.338602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dfb1f-eed3-4ef4-8926-f71426b8741e None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:55.339169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dfb1f-eed3-4ef4-8926-f71426b8741e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:55.339169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309dfb1f-eed3-4ef4-8926-f71426b8741e None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:55.340532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 748/1514] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:16:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:16:55.346058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefba65-ef51-4544-a66f-dff1551d8560 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:55.346533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefba65-ef51-4544-a66f-dff1551d8560 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:55.346643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aefba65-ef51-4544-a66f-dff1551d8560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:55.366039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2aefba65-ef51-4544-a66f-dff1551d8560 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:16:55.367468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2aefba65-ef51-4544-a66f-dff1551d8560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:16:55.376199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 767/1515] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:16:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:16:55.535740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dfb1f-eed3-4ef4-8926-f71426b8741e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:16:55.550228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309dfb1f-eed3-4ef4-8926-f71426b8741e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:16:55.557283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:16:55.557911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:55.558183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:55.558484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:55.567690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:16:55.569291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b8a4bba-ac52-4560-b32e-4f223534b83a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:16:55.580194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 749/1516] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:16:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:16:56.332819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ce386-5d2d-4bef-a306-2ebaed629f02 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:56.333081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ce386-5d2d-4bef-a306-2ebaed629f02 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:56.333365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-688ce386-5d2d-4bef-a306-2ebaed629f02 None None] PATH_INFO: `/v3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:56.334679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 768/1517] 10.222.0.75 () {54 vars in 925 bytes} [Tue Aug 30 04:16:56 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:16:56.341086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c69fb-786e-4b12-bcb1-706f5877bdd6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:16:56.341547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c69fb-786e-4b12-bcb1-706f5877bdd6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:16:56.342278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276c69fb-786e-4b12-bcb1-706f5877bdd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:16:56.401145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-276c69fb-786e-4b12-bcb1-706f5877bdd6 None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:16:56.402389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-276c69fb-786e-4b12-bcb1-706f5877bdd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:16:56.420940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:16:56.477704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 750/1518] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Aug 30 04:16:56 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:00.343205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f736-d615-449f-b83b-dbc03f456a28 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:00.343741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f736-d615-449f-b83b-dbc03f456a28 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:00.343741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b091f736-d615-449f-b83b-dbc03f456a28 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:00.345110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 769/1519] 10.222.0.46 () {54 vars in 961 bytes} [Tue Aug 30 04:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:17:00.349904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acaad-fa19-4888-bfab-402ad424addf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:00.350342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acaad-fa19-4888-bfab-402ad424addf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:00.350598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413acaad-fa19-4888-bfab-402ad424addf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:00.370718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-413acaad-fa19-4888-bfab-402ad424addf None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:00.371913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-413acaad-fa19-4888-bfab-402ad424addf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:00.380942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 751/1520] 10.222.0.46 () {58 vars in 1058 bytes} [Tue Aug 30 04:17:00 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:00.388478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f736-d615-449f-b83b-dbc03f456a28 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:00.402063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b091f736-d615-449f-b83b-dbc03f456a28 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:00.407180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:00.407770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:00.408049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:00.408320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:00.416442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:00.418002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a9f7fb-1895-4600-9008-77d59f249351 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:00.428200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 770/1521] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:00 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:06.837611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dc7b7-4367-4821-8892-dd21cdcf3f16 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:06.838093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dc7b7-4367-4821-8892-dd21cdcf3f16 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:06.838188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52dc7b7-4367-4821-8892-dd21cdcf3f16 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:06.839553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 752/1522] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:17:06.844802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300b1da-9db4-4e53-b28c-0642e1285a8c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:06.845237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300b1da-9db4-4e53-b28c-0642e1285a8c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:06.845346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5300b1da-9db4-4e53-b28c-0642e1285a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:06.864299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5300b1da-9db4-4e53-b28c-0642e1285a8c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:06.865488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5300b1da-9db4-4e53-b28c-0642e1285a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:06.873838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 771/1523] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:17:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:07.026433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dc7b7-4367-4821-8892-dd21cdcf3f16 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:07.041085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52dc7b7-4367-4821-8892-dd21cdcf3f16 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:07.047381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:07.047970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:07.048244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:07.048512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:07.056526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:07.058095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e402fd2-6bf8-42d0-90ba-3dfe2703fa56 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:07.068168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 753/1524] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:07.891778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4cd43-55c3-4bb7-a3f3-65e205d94360 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:07.892346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4cd43-55c3-4bb7-a3f3-65e205d94360 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:07.892346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a4cd43-55c3-4bb7-a3f3-65e205d94360 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:07.893990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 772/1525] 127.0.0.1 () {54 vars in 901 bytes} [Tue Aug 30 04:17:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:17:07.898707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a973d2-e6c3-4fde-9aa0-9b309df50c27 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:07.899120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a973d2-e6c3-4fde-9aa0-9b309df50c27 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:07.899252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a973d2-e6c3-4fde-9aa0-9b309df50c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:07.918657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d5a973d2-e6c3-4fde-9aa0-9b309df50c27 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:07.920464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d5a973d2-e6c3-4fde-9aa0-9b309df50c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:07.928415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 754/1526] 127.0.0.1 () {58 vars in 998 bytes} [Tue Aug 30 04:17:07 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:08.081699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4cd43-55c3-4bb7-a3f3-65e205d94360 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:08.096725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a4cd43-55c3-4bb7-a3f3-65e205d94360 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:08.102097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:08.102892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:08.103019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:08.103248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:08.111701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:08.113247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a137b21f-12f8-4841-8324-ae8785c2e0f6 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:08.122437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 773/1527] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:12.096901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:12.097487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:12.097487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:12.098843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 755/1528] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:17:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:17:12.103926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3db6-9775-4273-b7ac-5298b585af7e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:12.104340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3db6-9775-4273-b7ac-5298b585af7e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:12.104464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796c3db6-9775-4273-b7ac-5298b585af7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:12.124117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-796c3db6-9775-4273-b7ac-5298b585af7e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:12.125304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-796c3db6-9775-4273-b7ac-5298b585af7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:12.133035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 774/1529] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:17:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:12.424003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:12.425783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:12.505703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3a66f8-d512-433b-b9e2-a927c4466cea None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:12.509297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:12.509906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:12.510193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:12.510495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:12.519851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:12.521288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0a921c-0349-4edd-a17d-32c835ba1e59 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:12.530922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 756/1530] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:14.467808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17f571-73aa-472e-bdd7-c576f1c2fb77 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:14.468382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17f571-73aa-472e-bdd7-c576f1c2fb77 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:14.468382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a17f571-73aa-472e-bdd7-c576f1c2fb77 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:14.469793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 775/1531] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:17:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:17:14.475025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec695-4fb1-4052-bce7-b30066664eed None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:14.475471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec695-4fb1-4052-bce7-b30066664eed None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:14.475578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9ec695-4fb1-4052-bce7-b30066664eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:14.495258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6e9ec695-4fb1-4052-bce7-b30066664eed None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:14.496446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6e9ec695-4fb1-4052-bce7-b30066664eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:14.504248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 757/1532] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:17:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:14.798420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17f571-73aa-472e-bdd7-c576f1c2fb77 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:14.809036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a17f571-73aa-472e-bdd7-c576f1c2fb77 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:14.813495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:14.814089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:14.814401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:14.814669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:14.823939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:14.825522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d79e775-3520-48c5-9720-3ce0d0fe437d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:14.835001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 776/1533] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:14 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:18.304104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:18.304680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:18.304680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:18.306038 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 758/1534] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:17:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:17:18.311402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0f97-4741-4e4b-91f8-8ee7d04c38eb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:18.311820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0f97-4741-4e4b-91f8-8ee7d04c38eb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:18.311931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3c0f97-4741-4e4b-91f8-8ee7d04c38eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:18.331613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5d3c0f97-4741-4e4b-91f8-8ee7d04c38eb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:18.332806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5d3c0f97-4741-4e4b-91f8-8ee7d04c38eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:18.340576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 777/1535] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:17:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:18.896560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:18.898297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:18.963898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:17:18.977915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd22832-7091-480d-b1d9-11648cca493a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:18.981692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:18.982769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:18.983035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:18.983312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:18.992304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:18.993737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29dd29cc-8b25-41bd-977f-6333ebcf7a90 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:19.003994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 759/1536] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:20.972676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:20.973257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:20.973257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:20.974602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 778/1537] 127.0.0.1 () {54 vars in 917 bytes} [Tue Aug 30 04:17:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:17:20.979651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be0850-4400-4b84-a3d5-6c9363d336e9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:20.980050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be0850-4400-4b84-a3d5-6c9363d336e9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:20.980199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be0850-4400-4b84-a3d5-6c9363d336e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:21.000024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-17be0850-4400-4b84-a3d5-6c9363d336e9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:17:21.001223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-17be0850-4400-4b84-a3d5-6c9363d336e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:21.008994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 760/1538] 127.0.0.1 () {58 vars in 1014 bytes} [Tue Aug 30 04:17:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:17:21.577742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:17:21.579592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:17:21.587720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:17:21.594664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0c6ef02-ac45-4232-a4ed-312e7f33613f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:17:21.598160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:17:21.598776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:17:21.599055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:17:21.599348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:17:21.608355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:17:21.609781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732679a3-6149-4a48-b910-5717fc2087fb service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:17:21.619051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 779/1539] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:17:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:17:30.994844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=11 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:17:31.615023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:46.173881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d580-8133-48a6-ad25-5429d689e4ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.174556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d580-8133-48a6-ad25-5429d689e4ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.174855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd16d580-8133-48a6-ad25-5429d689e4ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.196980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffbbd-58ea-4a9d-8065-ad66364eee22 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.200868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bd16d580-8133-48a6-ad25-5429d689e4ff None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.201334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffbbd-58ea-4a9d-8065-ad66364eee22 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.201838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37ffbbd-58ea-4a9d-8065-ad66364eee22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.206850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bd16d580-8133-48a6-ad25-5429d689e4ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.233487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 761/1540] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.245585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a37ffbbd-58ea-4a9d-8065-ad66364eee22 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.251528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a37ffbbd-58ea-4a9d-8065-ad66364eee22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.259557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.259883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.260163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.266335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 780/1541] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:18:46.277001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.277632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.278103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.315436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.316678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.324375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.325862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.337589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 781/1542] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:18:46.343667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.356804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 762/1543] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:18:46.362677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.363339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e78b56-bcc7-4785-b3d2-419c9784a3cf None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.377863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.378512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.378787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.379052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.379702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:46.379997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.380902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8405f4a8-0075-409d-88e9-582235587e09 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.381037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:18:46.381037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:18:46.381037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:18:46.381870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89de7df3-57cb-494e-8412-c01bc92ca83a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.405016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.405646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.405926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.406200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.406899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:46.407317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.409002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e9d6a9-44a4-4d5f-a355-3ceb449d7438 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.409873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 782/1544] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:18:46.411452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.411743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.414358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.438873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 763/1545] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:46.440135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.440416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.440688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.464544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.465779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.472024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.473238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.488346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 783/1546] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.489184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.502977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 764/1547] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.503613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.516021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e643a88-6f81-45a4-a6fd-df0ea59ddb38 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.520984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6156ffb0-0d22-446d-aac2-ea824cbcd35c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.527365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.528231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.528510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.528779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.529427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.531363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59a1ea-f107-42f6-9aa9-83ffdfaf66d0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.535260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.535879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.536153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.536421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.537057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.538077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:18:46.538077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:18:46.538077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:18:46.540273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58153f91-3a08-47b5-9215-5940806e6127 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.582658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 784/1548] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/projects => generated 393 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:46.584058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58068-49d4-49ab-af01-f36869d6e97b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.584344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58068-49d4-49ab-af01-f36869d6e97b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.584619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a58068-49d4-49ab-af01-f36869d6e97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.593142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 765/1549] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/projects => generated 371 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:46.594531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d91fa-9eb4-4b78-81ae-37ae72f34717 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.594814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d91fa-9eb4-4b78-81ae-37ae72f34717 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.595085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1d91fa-9eb4-4b78-81ae-37ae72f34717 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.606526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-56a58068-49d4-49ab-af01-f36869d6e97b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.607774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-56a58068-49d4-49ab-af01-f36869d6e97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.622564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9d1d91fa-9eb4-4b78-81ae-37ae72f34717 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.623891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9d1d91fa-9eb4-4b78-81ae-37ae72f34717 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.676815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 785/1550] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.678371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddc2ac-5934-4845-bb8b-fcfa331bb337 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.678656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddc2ac-5934-4845-bb8b-fcfa331bb337 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.678927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ddc2ac-5934-4845-bb8b-fcfa331bb337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.685920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 766/1551] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.690730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb70f-8b5b-4dca-a6ec-a7f72aeceb8f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.690730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb70f-8b5b-4dca-a6ec-a7f72aeceb8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.690730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93deb70f-8b5b-4dca-a6ec-a7f72aeceb8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.704157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f4ddc2ac-5934-4845-bb8b-fcfa331bb337 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.705382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f4ddc2ac-5934-4845-bb8b-fcfa331bb337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.707304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-93deb70f-8b5b-4dca-a6ec-a7f72aeceb8f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.708455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-93deb70f-8b5b-4dca-a6ec-a7f72aeceb8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.713500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 786/1552] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.714897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.715180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.715449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.718330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 767/1553] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.724730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.725028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.725369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.734933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.736141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.744053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 787/1554] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.744845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.745462 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.746963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.755142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f80cbfb-1929-45ee-96a9-9a72d75c5065 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.755558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 768/1555] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.756276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.760040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.761407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.761855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.762313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.763112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.764299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:18:46.764299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:18:46.764299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:18:46.765496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.768127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a48c5e8b-69e1-41dc-923c-8d59e318cd39 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.772961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.773577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.774089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.774398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.775488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.778158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf3510b-3f35-4fd4-b94b-b4b9ed42718a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.790279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 788/1556] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/users => generated 297 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:46.790279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.800840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a10b014-4266-4ec8-b88f-836c2671c860 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.804980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 769/1557] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/users => generated 286 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:46.805507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.806109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.806319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14cda-198b-4505-b0e8-8505e274459d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.806466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.806599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14cda-198b-4505-b0e8-8505e274459d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.806705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.806865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c14cda-198b-4505-b0e8-8505e274459d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.807304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:46.808237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.809904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.825666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 789/1558] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:46.826795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.827679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-76c14cda-198b-4505-b0e8-8505e274459d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.828889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-76c14cda-198b-4505-b0e8-8505e274459d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.838331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 770/1559] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.839811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b996-27b2-4dd8-b86a-b0e5d045465d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.840095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b996-27b2-4dd8-b86a-b0e5d045465d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.840369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a4b996-27b2-4dd8-b86a-b0e5d045465d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.842820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8d529e-d10c-4f56-8295-ca9a36658182 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.847391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.847984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.848259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.848524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.849154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:46.849449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.851039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-203848d7-ca9e-4908-bb68-8d8163ee54f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.861028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35a4b996-27b2-4dd8-b86a-b0e5d045465d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.862231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35a4b996-27b2-4dd8-b86a-b0e5d045465d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.863181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 790/1560] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:46.864376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.864698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.864921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.871852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 771/1561] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.873471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.873853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.875225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.885009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.886792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.895801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 791/1562] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.896407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:46.896584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.897589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:46.906201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 772/1563] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:46.906423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92fac5-64be-4990-b57f-ea635772fb49 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.907056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.911607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.912211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.912486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.912757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.913395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.915044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.919639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e3c310-cf5d-4489-a401-af634f4ba3d7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.924452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.925076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.925354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.925629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.926299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.930482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 792/1564] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:46.930482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.930809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.942691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 773/1565] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:46.942691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.942950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2647d0bb-16d3-4403-b84d-0a49df3a4aaf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.945461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.946065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.946375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.946643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.947279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.949201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.952825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9696a2-ce3b-46e6-89bb-4e19caa3df07 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:46.957485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:46.958084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:46.958401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:46.958676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:46.959348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:46.959647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:46.961242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:46.971211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:46.975127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 774/1566] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:46.975841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.979111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 793/1567] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:46.980953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:46.988877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c7668a3-e50f-45b0-aa07-1a7ceb57d2e0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.007615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:47.045113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041c1d82-9811-4b37-912c-f5698e2bd15c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.048623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.049236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.049507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.049799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.049973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.050445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.050626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.050907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.051179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.051822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:47.052124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.052367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.053690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.066122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 794/1568] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:47.071208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.080249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 775/1569] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:46 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.081166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.082165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac05c8b-b83c-49f0-803d-2acdd7c9cc9c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.140414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.141676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.141965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.142589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.143310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:47.143640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.145231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.146843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e9cab2-6631-446b-848e-4af53d7fdd3d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.152290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.152888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.153160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.153424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.154062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:47.154393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.155977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.158477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 795/1570] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:47.159292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.169551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e943254-7bcc-4f8a-aac3-3191ec754ff1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.173322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 776/1571] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:47.173322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.176315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.176910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.177192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.177462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.178095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.179711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.188360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77959dcf-5800-46db-b339-5eb5cd3c0068 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.190786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 796/1572] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.191572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.193570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.194204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.194624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.194897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.195544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.197168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.204589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330839-c8f4-42e5-9cd6-7b82a10fdbd4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.207950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 777/1573] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.208731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.209645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.210433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.210798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.211480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.212562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.215833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.219568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa82926-8215-46e6-b134-7ea568638960 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.224181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.224772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.225049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.225323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.225948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.227888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.248989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 797/1574] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.248989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.256199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 778/1575] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 287 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.256199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.270898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-447b7c90-b690-4757-add5-8fb9d23bf036 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.306921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:47.315572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4838452-32b1-4a16-a896-dbc92afe0262 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.320366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.320980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.321472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.321749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.322439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.325349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.338493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.339655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.339948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.340188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.340810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.342889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.352672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 798/1576] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/projects => generated 377 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.353490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.369286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 779/1577] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 295 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.371466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.376989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:47.382154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c37f1f-afd9-4de9-a788-9da1bbf9fe4d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.421028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8398e71-59c0-490a-8bd0-2cf9957fb00b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.426077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.426717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.426997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.427268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.427962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.429878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.444405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.444936 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.445418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.445694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.446697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.448496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.462170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 799/1578] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/projects => generated 405 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.463037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.480698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 780/1579] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/projects => generated 375 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.480698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.529176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c023131e-c017-4e40-93d6-c2e9d632bc96 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.551245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf02a63c-d349-46a9-87ae-7aa474ec0cec admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.555930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.556583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.556858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.557124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] PATH_INFO: `/v3/projects/ea96b9dd52f143de85234abcdb3e93a0/users/1db869b85397448bb15ff45866fea7f6/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.572589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea96b9dd52f143de85234abcdb3e93a0, user_id=1db869b85397448bb15ff45866fea7f6, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.573710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:18:47.573710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:18:47.573710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:18:47.575677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.589102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.589897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 781/1580] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:47 2022] PUT /identity/v3/projects/ea96b9dd52f143de85234abcdb3e93a0/users/1db869b85397448bb15ff45866fea7f6/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:47.590045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.590338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.590691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] PATH_INFO: `/v3/projects/a9a5c650ed23445bb585711f962d7e09/users/eb61948f41344ea481024a96cf03db37/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.590790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.606978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9a5c650ed23445bb585711f962d7e09, user_id=eb61948f41344ea481024a96cf03db37, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.609822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.612866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:47.626308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 800/1581] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:47 2022] PUT /identity/v3/projects/a9a5c650ed23445bb585711f962d7e09/users/eb61948f41344ea481024a96cf03db37/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:47.627203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.665093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d00eb3-b8dd-408a-9798-b3bed5ecc8bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.678336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:47.691398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9724b0-37c5-4843-9621-5737c46b4f6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.696107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.696658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.696934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.697201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.697838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.699464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.709421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 801/1582] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.710201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.721487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccaff5a-3554-47fb-8eb0-dd518c858fb1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.726386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.726974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.727243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.727386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.727511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.727860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.728133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.728250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.728408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.729036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.730792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.731852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.754453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 782/1583] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 286 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.754864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 802/1584] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 289 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.755369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.756105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.766586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af42839f-b29a-446c-9131-8c0231465a4e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.767537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782f0e7a-49c4-4641-9cd0-297c96da1f3e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.771406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.771996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.772140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.772283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.772555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.772692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.772976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.773175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.773281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.773926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.775271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fac263-5a73-4c21-b747-6793d0e11efc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.775546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.785898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 783/1585] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.786828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.797375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6819b7e-ea77-4f9d-a2f1-afa6439a71ba admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.803315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 803/1586] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 303 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.803315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.803315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.803315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.805510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.806111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.806443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.806717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.807346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.809817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5772df69-5d9c-49a5-89a6-92a5e2a6cce9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.832654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 784/1587] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/users => generated 288 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:47.833023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] MFA Rules not processed for user `1db869b85397448bb15ff45866fea7f6`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:47.834287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:47.834630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.834918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.835340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.854074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:47.855485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:47.863060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 785/1588] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:47.863904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.873607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9611ed3-67b9-4209-b05f-358892b5fa3e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.878208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.878830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.879107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.879372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.880007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.881603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.892946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 786/1589] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.896678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.903642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f2d298-e340-4c4a-b592-36692008fb70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.908155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.908730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.909006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.909277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.909900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.910898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 804/1590] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:47.911520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.911798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.927972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 787/1591] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.927972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.929529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a876c4-6ec5-4c6d-8051-4ca41243a054 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.935651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.936286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.936724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.937077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.937930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.939564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.940106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2143ec1-e44e-4cc0-be54-cc896b7e0190 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.946324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.947189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.947766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.947766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.949331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.951259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.951384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 805/1592] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.955283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.966715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 788/1593] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.966715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.966981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2d23a8-1479-4e92-9080-7e792080427b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.969876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.970506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.970792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.971060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.971703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.973289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.974397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf15ad73-186d-499b-9358-b0170c1c0e55 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:47.979678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:47.980288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:47.980545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:47.980813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:47.981446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:47.983067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:47.984025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 806/1594] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.984830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:47.996771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 789/1595] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:47.997557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.007312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abb1e4f-5053-45ca-972e-7d9900494e5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.012285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.012922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.013201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.013473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] PATH_INFO: `/v3/projects/58eb699706ee413da3afe8df3ad5d2d7/users/65da7cfb074945528c7d81268d14d57f/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.013583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.029891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58eb699706ee413da3afe8df3ad5d2d7, user_id=65da7cfb074945528c7d81268d14d57f, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.032593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.045542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf551a-34f4-4fb1-902e-c63e623a287f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.056632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 790/1596] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:47 2022] PUT /identity/v3/projects/58eb699706ee413da3afe8df3ad5d2d7/users/65da7cfb074945528c7d81268d14d57f/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.056632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.106779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.107575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.107886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.108159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.110592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ebf155-72b8-48bf-a90c-4330435a1dc4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.115353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.115954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.116232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.116504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.117144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.117388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.118790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.118928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fe0842-eca4-4c4f-a6a0-5357286b1de3 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.124427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.129099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 807/1597] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:48.130608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef418-e007-4dc7-82f7-31d227181831 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.130956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef418-e007-4dc7-82f7-31d227181831 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.131235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 791/1598] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:48.131369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78ef418-e007-4dc7-82f7-31d227181831 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.132719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 808/1599] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 30 04:18:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:18:48.132913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.133712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef418-e007-4dc7-82f7-31d227181831 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.144210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a0e342-3a26-42f1-81e9-6e89c7ddd04e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.147724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78ef418-e007-4dc7-82f7-31d227181831 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.148884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.149524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.149801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.150100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.150784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.153273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.155571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.156588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.157037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.157467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] PATH_INFO: `/v3/projects/411b8da28bed456bbdcdb01587610b47/users/8c30ec9810204a4685590c42707e41f8/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.165413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 792/1600] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:48.166375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.177604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8223d6e6-4ace-4abb-aff7-c552b0e213c1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.178438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] RBAC: Authorizing `identity:create_grant(project_id=411b8da28bed456bbdcdb01587610b47, user_id=8c30ec9810204a4685590c42707e41f8, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.181423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6b07b5-3819-46a3-b13a-6c0ee9ea911f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.182663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.183455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.183630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.183880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.184566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.186385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.197948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 809/1601] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:48 2022] PUT /identity/v3/projects/411b8da28bed456bbdcdb01587610b47/users/8c30ec9810204a4685590c42707e41f8/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.201821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 793/1602] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:48.202703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.204178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f59-9f2a-4c6f-a2de-c864298f5f37 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.204557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f59-9f2a-4c6f-a2de-c864298f5f37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.204942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19289f59-9f2a-4c6f-a2de-c864298f5f37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.213244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d6595d-bf47-441d-ba96-b2472f36b488 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.237349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-19289f59-9f2a-4c6f-a2de-c864298f5f37 None None] MFA Rules not processed for user `65da7cfb074945528c7d81268d14d57f`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.239312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-19289f59-9f2a-4c6f-a2de-c864298f5f37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.281371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.281982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.282269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.282559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.283513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.285134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.301151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 794/1603] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:48.301151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.316550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93133faf-8217-494d-86f6-3b0cc7263abd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.323371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.323775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.324124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.324490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] PATH_INFO: `/v3/projects/fd0f64a24a554a74a3b21e37aaf7e966/users/bdce1a7433a344599533213dc17f4878/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.329300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 810/1604] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.330418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.330985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.331457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.359111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd0f64a24a554a74a3b21e37aaf7e966, user_id=bdce1a7433a344599533213dc17f4878, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.362356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.380867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 795/1605] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:48 2022] PUT /identity/v3/projects/fd0f64a24a554a74a3b21e37aaf7e966/users/bdce1a7433a344599533213dc17f4878/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.383521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.388909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.390238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.429403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.470993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599c5b07-72b2-447f-89de-0a4917a7671b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.475885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.476696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.477100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.477527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] PATH_INFO: `/v3/projects/5a89b6f21ff24eafbed378c2ece79c3e/users/96d18f6efa1f4c91a6f213c095c507a3/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.478421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 811/1606] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.480335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.493414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32404ab9-88b2-47b4-9e03-2a3916d217ea None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.498658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a89b6f21ff24eafbed378c2ece79c3e, user_id=96d18f6efa1f4c91a6f213c095c507a3, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.499118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.499822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.500101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.500376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] PATH_INFO: `/v3/projects/8be1dd6cbfc544c4b430b2420bb6bdca/users/47f52ec04d454dff861474eb9eb09d5f/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.502914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a18fcb-9fbf-4d66-9788-d1969397f466 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.518734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8be1dd6cbfc544c4b430b2420bb6bdca, user_id=47f52ec04d454dff861474eb9eb09d5f, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.519728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 796/1607] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:48 2022] PUT /identity/v3/projects/5a89b6f21ff24eafbed378c2ece79c3e/users/96d18f6efa1f4c91a6f213c095c507a3/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.522113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.524744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.525090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.525444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.539209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 812/1608] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:48 2022] PUT /identity/v3/projects/8be1dd6cbfc544c4b430b2420bb6bdca/users/47f52ec04d454dff861474eb9eb09d5f/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.540458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.548637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.549846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.586229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.627057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 797/1609] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.628289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.638733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa378984-e816-40ec-b854-1f5efabea226 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.640589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196a3dc7-0352-4646-b87c-47251ee144b3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.646798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.647411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.647686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.647956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.648944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.650315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.651323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e3b3bd-2bfe-4f01-9f97-99173d5f8ef5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.653151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.653618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.654131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] PATH_INFO: `/v3/projects/fe278e07fd7940dca53f7c2b8b792326/users/b0e68c885560485ead1c61fe83fc3ce4/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.676745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe278e07fd7940dca53f7c2b8b792326, user_id=b0e68c885560485ead1c61fe83fc3ce4, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.681076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.687003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 798/1610] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/projects => generated 373 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:48.688466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146973-ac29-402d-8c40-a872846d3abc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.688808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146973-ac29-402d-8c40-a872846d3abc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.689082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b146973-ac29-402d-8c40-a872846d3abc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.706429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 813/1611] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:48 2022] PUT /identity/v3/projects/fe278e07fd7940dca53f7c2b8b792326/users/b0e68c885560485ead1c61fe83fc3ce4/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:48.707855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.731384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0b146973-ac29-402d-8c40-a872846d3abc None None] MFA Rules not processed for user `bdce1a7433a344599533213dc17f4878`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.734954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0b146973-ac29-402d-8c40-a872846d3abc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.804683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3828a50-a687-4af0-bc5f-f738be0fd7ca admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.812217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.812935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.813765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.813765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.815996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 799/1612] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.819482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.819482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.819482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.839149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.840843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.842927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.886528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.888665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fa50f63-e188-4135-b43f-406328b3d6f7 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.898349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 814/1613] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:48.900079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.900502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.900875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.918830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 800/1614] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.920576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.935533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:48.936898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:48.958448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 815/1615] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:48.960343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:48.975908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5391c29-62d1-44c1-96a2-9376ccdc3684 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:48.981678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:48.983999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:48.985089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:48.985664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:48.985939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:48.988032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:48.991168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd238f3-9499-47a5-af29-f7a9168d9990 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:48.999914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf0836-4897-44cc-a109-0de199006ecf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.005138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.006197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.006608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.007080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.025931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 816/1616] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:48 2022] POST /identity/v3/users => generated 293 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:49.027295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.027624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.027957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.028623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.030222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.053662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 801/1617] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:48 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:49.055261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.065501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8709635-1428-4ec2-870d-17380b907f84 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.069676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:49.070608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.071013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:49.071389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.071771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.072148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.072888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.075719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.091493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 817/1618] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:49.094190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.119578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f7aa478-649b-4890-b595-14599bdec527 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.125798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.126778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.127125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.127402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.139744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.141342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.153927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 802/1619] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:49 2022] POST /identity/v3/projects => generated 371 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:49.156372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 818/1620] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:49.157017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.157246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.170621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8c33c0b-8748-47f9-bc4d-b573c06281c2 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.239524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.240211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.240511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.240983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.241649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.243514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.250937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee0183f-4cc9-4d02-8c0c-83b958a18bc5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.258601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 819/1621] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:49.258798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.258939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.259055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.259055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.259055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.259303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:49.272061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.273413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e251fb3-f193-411b-ad89-1c395385a373 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.273980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.279314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.280342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.280733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.281117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.288690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 803/1622] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:49.289895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.295010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.296940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fbd142-8562-4c34-be70-3ccfddf24b8f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.305733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7be548-7219-49f3-8168-076dcc52e997 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.309880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 820/1623] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:49.311560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282fad-d0dc-4670-93b5-10bb0c60c1c6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.311847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282fad-d0dc-4670-93b5-10bb0c60c1c6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.312122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f282fad-d0dc-4670-93b5-10bb0c60c1c6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.313764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 821/1624] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 30 04:18:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:18:49.314674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f282fad-d0dc-4670-93b5-10bb0c60c1c6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.333340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f282fad-d0dc-4670-93b5-10bb0c60c1c6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.333932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.334750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.335524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.335951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.336689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.341153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.342948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.342948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.343158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] PATH_INFO: `/v3/projects/2679606b01e64586a5bb07934ec54ea2/users/8f60022a120a476f94c73317b9804928/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.343504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.362476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2679606b01e64586a5bb07934ec54ea2, user_id=8f60022a120a476f94c73317b9804928, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.365399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106328b-b48c-4cfe-bf2a-988da70f7f98 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.386795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 822/1625] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:49 2022] PUT /identity/v3/projects/2679606b01e64586a5bb07934ec54ea2/users/8f60022a120a476f94c73317b9804928/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:49.388603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.388967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.389326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.398562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 804/1626] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:49 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:49.398898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.414527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:49.416073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:49.425802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38348c3c-4551-46c8-9b7f-d74f9d822d66 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.456401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:49.519909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 823/1627] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:18:49 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:49.521223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.607882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.608695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.610285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.610453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.611056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.615407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.617070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fc3cb1-51e9-4665-a461-34c56bd03f44 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.624557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.625893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.626515 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.627175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.628064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.630647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.647933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 805/1628] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:49.648739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.650647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 824/1629] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:49.653412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.665854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537388f2-89f4-4a1c-b841-24951be99582 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.671424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.673200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.673758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.674186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] PATH_INFO: `/v3/projects/2679606b01e64586a5bb07934ec54ea2/users/8f60022a120a476f94c73317b9804928/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.676494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2679606b01e64586a5bb07934ec54ea2, user_id=8f60022a120a476f94c73317b9804928, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.679604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.706342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 825/1630] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:49 2022] PUT /identity/v3/projects/2679606b01e64586a5bb07934ec54ea2/users/8f60022a120a476f94c73317b9804928/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:49.707446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:49.751744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:49.771265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245651c0-cf4d-44c9-8c22-11745c851334 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.847176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e85ffe-a5f2-447d-bb89-ccd0f664f764 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:49.855978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.856671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.857116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.857403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] PATH_INFO: `/v3/projects/3720f53142dd4af69c9df022291cb870/users/d21ef229781247f1a930a26367519741/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.874738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:49.875495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.875866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.876393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.881077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3720f53142dd4af69c9df022291cb870, user_id=d21ef229781247f1a930a26367519741, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:49.884756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-326f416a-fe72-43e9-aa24-c6d9dd90a05e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:49.935612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 826/1631] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:49 2022] PUT /identity/v3/projects/3720f53142dd4af69c9df022291cb870/users/d21ef229781247f1a930a26367519741/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:49.937420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:49.938540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:49.941699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:49.998496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] MFA Rules not processed for user `8f60022a120a476f94c73317b9804928`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:49.998753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:50.000501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:50.001407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:50.016544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 806/1632] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:49 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 370 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:50.017815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:50.037803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:50.064416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267e62f9-eb3a-4f84-977d-7f6fca28fff3 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:50.188537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 827/1633] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:49 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:50.202580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:50.221733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:50.224785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:50.225169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:50.225542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:50.228425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:50.230321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:50.258738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 807/1634] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:50 2022] GET /identity/v3/roles => generated 1420 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:50.261538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:50.289877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f874642-e472-4bb6-9125-0f5aac7a74de admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:50.304270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:50.306411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:50.306799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:50.307222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] PATH_INFO: `/v3/projects/3720f53142dd4af69c9df022291cb870/users/d21ef229781247f1a930a26367519741/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:50.314947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3720f53142dd4af69c9df022291cb870, user_id=d21ef229781247f1a930a26367519741, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:50.318270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210d08d5-74aa-4b82-a833-a055c92ee9e7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:50.331267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8ea2b9-1dbf-4fa1-9650-2526bceeac99 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:50.345497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 808/1635] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:50 2022] PUT /identity/v3/projects/3720f53142dd4af69c9df022291cb870/users/d21ef229781247f1a930a26367519741/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:18:50.356561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:50.356985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:50.357353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:50.437485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] MFA Rules not processed for user `d21ef229781247f1a930a26367519741`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:50.439050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:50.471977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:50.481004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:50.484291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:50.484589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:50.484870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:50.609241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 809/1636] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:50.687309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:50.689035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:50.710322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 828/1637] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:50 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 508 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:18:50.818373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:50.832174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94a88b0-7132-4fcf-a92b-5c9f2f56588c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:50.837364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:50.838379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:50.839063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:50.839495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:50.858238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:50.859968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e51e0570-3e79-4b40-9331-3e64386a7b8a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:50.900267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 810/1638] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:51.147321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:51.165458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6b7e28-d936-4440-9561-a3a14d880c7b service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:51.177717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:51.178694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:51.179178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:51.179550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:51.256062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:51.334769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:51.337797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e703a8-4f36-4a80-b71b-e3a6070c9050 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:51.361873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 829/1639] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:51.982698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d978-8005-41ad-9d48-55f3f3968bb6 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:51.982698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d978-8005-41ad-9d48-55f3f3968bb6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:51.983435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94d978-8005-41ad-9d48-55f3f3968bb6 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:51.984682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 811/1640] 10.222.0.46 () {54 vars in 962 bytes} [Tue Aug 30 04:18:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:18:51.994672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d738-f059-4919-b780-cb3b511ee025 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:51.995664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d738-f059-4919-b780-cb3b511ee025 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:51.996183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8757d738-f059-4919-b780-cb3b511ee025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.048694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8757d738-f059-4919-b780-cb3b511ee025 None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:52.050617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8757d738-f059-4919-b780-cb3b511ee025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:52.119260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d978-8005-41ad-9d48-55f3f3968bb6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:52.214277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 830/1641] 10.222.0.46 () {58 vars in 1059 bytes} [Tue Aug 30 04:18:51 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 220 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:52.219739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614dae5-97ef-465b-a152-386b5c51d0c4 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:52.220109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614dae5-97ef-465b-a152-386b5c51d0c4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:52.220457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1614dae5-97ef-465b-a152-386b5c51d0c4 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.222204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 831/1642] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 30 04:18:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:18:52.230128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:52.230811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:52.231222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.279075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94d978-8005-41ad-9d48-55f3f3968bb6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:52.279537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:52.281060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:52.290365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:52.292212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:52.293420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:52.295628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] PATH_INFO: `/v3/users/65da7cfb074945528c7d81268d14d57f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.298199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] RBAC: Authorizing `identity:delete_user(user_id=65da7cfb074945528c7d81268d14d57f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:52.299370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: WARNING py.warnings [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:18:52.299370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: warnings.warn(msg) Aug 30 04:18:52.299370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: Aug 30 04:18:52.302086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:52.305623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 832/1643] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:18:52.319906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:52.333041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ab150-e436-4930-a931-08f8455727c0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:52.353566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:52.353566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:52.353566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:52.353566 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.373781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:52.375217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adeac8a-0268-4ea2-b849-22ace718c51c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:52.400196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 833/1644] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:18:52.424981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '65da7cfb074945528c7d81268d14d57f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:52.437725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '65da7cfb074945528c7d81268d14d57f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:52.448916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '65da7cfb074945528c7d81268d14d57f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:52.450119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '65da7cfb074945528c7d81268d14d57f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:52.487068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '65da7cfb074945528c7d81268d14d57f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:52.501400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 812/1645] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:18:52 2022] DELETE /identity/v3/users/65da7cfb074945528c7d81268d14d57f => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:52.503761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:52.516588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:52.516588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:52.535455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:52.538716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:52.647962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 834/1646] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:52.852164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:52.853686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:52.891146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:53.001406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f6b6f2-2c68-4f5a-9633-a1ee838f0b65 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:53.017585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:53.018350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:53.018689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:53.019021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:53.041492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:53.043361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b2790a-e75a-4650-8f8e-a98f53f170fa service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:53.075281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 813/1647] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:53.917644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:53.920004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:53.935259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede49292-ab7c-40ca-83d2-fa39e4290693 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:53.940675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:53.941337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:53.941666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:53.941941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] PATH_INFO: `/v3/users/1db869b85397448bb15ff45866fea7f6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:53.943672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1db869b85397448bb15ff45866fea7f6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:53.945723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:53.991157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:54.012372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1db869b85397448bb15ff45866fea7f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:54.033019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1db869b85397448bb15ff45866fea7f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:54.033841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1db869b85397448bb15ff45866fea7f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:54.034643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1db869b85397448bb15ff45866fea7f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:54.045856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1db869b85397448bb15ff45866fea7f6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:54.075618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 835/1648] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:18:53 2022] DELETE /identity/v3/users/1db869b85397448bb15ff45866fea7f6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:54.079990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0888-589b-4b13-8fe5-839edd65c087 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:54.082085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0888-589b-4b13-8fe5-839edd65c087 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:54.083790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eca0888-589b-4b13-8fe5-839edd65c087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:54.124355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2eca0888-589b-4b13-8fe5-839edd65c087 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:54.131280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2eca0888-589b-4b13-8fe5-839edd65c087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:54.268294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 814/1649] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:54.282550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:54.284424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:54.372030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207668dd-baaf-4975-8ef8-24b9663c73c2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:54.376610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:54.377917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:54.378424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:54.379686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:54.394134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:54.396203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:54.430819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 836/1650] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:55.526925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:55.527867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:55.527867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:55.614705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] MFA Rules not processed for user `47f52ec04d454dff861474eb9eb09d5f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:55.616455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:55.827764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 815/1651] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:55 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 304 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:55.829115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:55.837323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:55.888083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:55.889790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:55.892487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c13f1d-8901-413d-b0a0-27d6a19afdc8 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:55.904576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:55.910310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:55.918055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d44ab1c-b116-467e-8137-2b987d1094a1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:55.922596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:55.922981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:55.925046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:55.930135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:55.933507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:55.937502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:55.937502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:55.937502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] PATH_INFO: `/v3/projects/58eb699706ee413da3afe8df3ad5d2d7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:55.938746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58eb699706ee413da3afe8df3ad5d2d7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:55.941135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:56.009987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 837/1652] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:55 2022] POST /identity/v3/projects => generated 405 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:56.014851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:56.101301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:56.115029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58eb699706ee413da3afe8df3ad5d2d7', 'request_id': 'req-468a46df-db1f-463e-9882-7afdfa87cba4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.137275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58eb699706ee413da3afe8df3ad5d2d7', 'request_id': 'req-468a46df-db1f-463e-9882-7afdfa87cba4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.155201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58eb699706ee413da3afe8df3ad5d2d7', 'request_id': 'req-468a46df-db1f-463e-9882-7afdfa87cba4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.216675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce412a8-9463-423c-b542-b05388b26b87 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:56.220761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] The token cache is being invalidate because project 58eb699706ee413da3afe8df3ad5d2d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:18:56.221691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.226443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.240561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 816/1653] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:18:55 2022] DELETE /identity/v3/projects/58eb699706ee413da3afe8df3ad5d2d7 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:56.242494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:56.256815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:56.383896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:56.457392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:56.459678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:56.460763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:56.463179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:56.463179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:56.469963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:56.507545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-468a46df-db1f-463e-9882-7afdfa87cba4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:56.526532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 838/1654] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:56 2022] POST /identity/v3/users => generated 309 bytes in 512 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:18:56.527896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:56.536242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:56.537560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:56.538774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:56.540844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:56.541261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] PATH_INFO: `/v3/projects/ea96b9dd52f143de85234abcdb3e93a0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:56.546379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea96b9dd52f143de85234abcdb3e93a0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:56.551993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:56.557533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c0c18-cca9-42bb-be7b-31bb932b2227 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:56.563742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:56.564521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:56.565235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:56.565235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] PATH_INFO: `/v3/users/8f60022a120a476f94c73317b9804928` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:56.566742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8f60022a120a476f94c73317b9804928)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:56.569413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:56.678481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8f60022a120a476f94c73317b9804928'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.709977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea96b9dd52f143de85234abcdb3e93a0', 'request_id': 'req-956bd50a-7666-4fdc-88f6-e91a90575530'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.722394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8f60022a120a476f94c73317b9804928'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.723734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8f60022a120a476f94c73317b9804928'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.726080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea96b9dd52f143de85234abcdb3e93a0', 'request_id': 'req-956bd50a-7666-4fdc-88f6-e91a90575530'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.728656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8f60022a120a476f94c73317b9804928'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.739777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea96b9dd52f143de85234abcdb3e93a0', 'request_id': 'req-956bd50a-7666-4fdc-88f6-e91a90575530'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.752816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8f60022a120a476f94c73317b9804928'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.780969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] The token cache is being invalidate because project ea96b9dd52f143de85234abcdb3e93a0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:18:56.781745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.783552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 839/1655] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:18:56 2022] DELETE /identity/v3/users/8f60022a120a476f94c73317b9804928 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:56.795278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-956bd50a-7666-4fdc-88f6-e91a90575530 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:56.800859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:56.801732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 817/1656] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:18:56 2022] DELETE /identity/v3/projects/ea96b9dd52f143de85234abcdb3e93a0 => generated 0 bytes in 560 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:56.807377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d7e8-3e86-43e4-ae07-66e0e0228682 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:56.809018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:56.809309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d7e8-3e86-43e4-ae07-66e0e0228682 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:56.810496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414d7e8-3e86-43e4-ae07-66e0e0228682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:56.870500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2414d7e8-3e86-43e4-ae07-66e0e0228682 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:56.873853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2414d7e8-3e86-43e4-ae07-66e0e0228682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:57.001405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3eab72-74f6-44d0-bfff-1ece6d93547a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.014989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.015884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.022757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.023198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.024018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.026116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.051123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 818/1657] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:57.053211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.053744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.054171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.058929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 840/1658] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:56 2022] GET /identity/v3/roles => generated 1420 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:57.061121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.063838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:57.065677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:57.095444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d412b9-3d88-4d63-a46e-6a15b9c20dea admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.096316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:57.101738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:57.119311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.120546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.121001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.121260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] PATH_INFO: `/v3/projects/d7f48b5d5da647cba4bf5d0622348383/users/9c354bbd0e7a414795f4e349f57d6a9a/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.121557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 819/1659] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:57.125943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.141227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e025da-559b-48d7-9267-9e1ddcde9e93 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.152032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.154006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.155012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.155672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.157034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:57.157814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.160499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.162300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f48b5d5da647cba4bf5d0622348383, user_id=9c354bbd0e7a414795f4e349f57d6a9a, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.167466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.192614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 820/1660] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:57.199401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.206312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 841/1661] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:57 2022] PUT /identity/v3/projects/d7f48b5d5da647cba4bf5d0622348383/users/9c354bbd0e7a414795f4e349f57d6a9a/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:57.207813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.214470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fc8f3f-6286-4dd5-9eef-c9b2e9826d2d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.241400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:57.344948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3022be44-9de4-43a4-a909-7a82cab0862b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.355993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.356657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.356902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.357171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.357828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.359476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.379977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.381331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.381751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.382180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.382959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 842/1662] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:57.383769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.387678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.389132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.414206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642075a5-6d4b-4097-8f9d-390de79704c5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.427152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.427828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.428107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.428373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] PATH_INFO: `/v3/projects/d7f48b5d5da647cba4bf5d0622348383/users/9c354bbd0e7a414795f4e349f57d6a9a/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.438386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7f48b5d5da647cba4bf5d0622348383, user_id=9c354bbd0e7a414795f4e349f57d6a9a, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.440727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.470728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 843/1663] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:57 2022] PUT /identity/v3/projects/d7f48b5d5da647cba4bf5d0622348383/users/9c354bbd0e7a414795f4e349f57d6a9a/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:57.508787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 821/1664] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:57 2022] POST /identity/v3/projects => generated 367 bytes in 311 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:57.509522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.672900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f8be02-706e-4e04-bcc9-74d83685f26e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.799538 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.800326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.800715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.801128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.801904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.805608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.845862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 844/1665] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:57 2022] POST /identity/v3/users => generated 284 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:57.849827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.869386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8e1380-fb77-46a8-bd8d-3a5b8a81720b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.880973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.881675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.881862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.882241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.882920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.888795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.906530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 822/1666] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:57.911342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.924859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5a0e13-39c3-4099-91f7-8874f476a060 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.937960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.938755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.939033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.939306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:57.940105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:57.941772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-329221a5-322d-4c09-9f30-b9d82b9ac3f1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:57.955300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 845/1667] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:57 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:57.959915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:57.979327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2d18b7-da35-4fd8-a925-4e20e719d3ba admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:57.985310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:57.986060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:57.988066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:57.988066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] PATH_INFO: `/v3/projects/a9569ed0183d40ebbc08c543421178be/users/98e69f7641f8431889369d108326b742/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.016249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] RBAC: Authorizing `identity:create_grant(project_id=a9569ed0183d40ebbc08c543421178be, user_id=98e69f7641f8431889369d108326b742, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:58.020106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:58.054150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:58.054677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:58.056501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.087788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 823/1668] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:57 2022] PUT /identity/v3/projects/a9569ed0183d40ebbc08c543421178be/users/98e69f7641f8431889369d108326b742/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:58.125905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] MFA Rules not processed for user `98e69f7641f8431889369d108326b742`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:58.128024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:58.141676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:58.284630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:58.288477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:58.316882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 846/1669] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:58.325379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:58.330200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:58.483256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-418c56c9-ab85-4e23-9285-5285a40ce09d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:58.495438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:58.496258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:58.496539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:58.496808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] PATH_INFO: `/v3/projects/2679606b01e64586a5bb07934ec54ea2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.499470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] RBAC: Authorizing `identity:delete_project(project_id=2679606b01e64586a5bb07934ec54ea2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:58.503026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:58.507512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493c6c8d-9cb8-4791-a335-afd7327b78e0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:58.524135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:58.524790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:58.525072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:58.525342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.549129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:58.550659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057b332c-e077-4486-926b-4fab44eae364 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:58.577414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 847/1670] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:18:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:18:58.594032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:58.623487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2679606b01e64586a5bb07934ec54ea2', 'request_id': 'req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:58.639126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2679606b01e64586a5bb07934ec54ea2', 'request_id': 'req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:58.655527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2679606b01e64586a5bb07934ec54ea2', 'request_id': 'req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:58.685367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] The token cache is being invalidate because project 2679606b01e64586a5bb07934ec54ea2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:18:58.686340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:58.689968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cc591221-cc4e-42b6-bafb-f3e7e7776e8f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:18:58.699597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:58.699966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:58.700333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.703762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 824/1671] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:18:58 2022] DELETE /identity/v3/projects/2679606b01e64586a5bb07934ec54ea2 => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:58.744747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:58.748273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:58.940356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 848/1672] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:58.951457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:58.951913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:58.952048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:58.985174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:58.993081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:59.021897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.023535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 825/1673] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:59.043131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25687f0e-c1cc-47c3-8a46-2cb1c1aaf1b9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.070559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.071752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.071752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.071752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.072378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:18:59.072679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.075023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.112469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 849/1674] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:18:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:18:59.113683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.128729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8d03c5-df25-4b48-85d7-ea5ca65ffd70 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.140071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.140706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.140991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.141261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.141906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.144154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.241930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 826/1675] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:59 2022] POST /identity/v3/projects => generated 363 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:59.248838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.264841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450d3ddc-b660-4021-a88d-28228597e36a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.370959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:18:59.455797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.455797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.455797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.455797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.456149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.456386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.503463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 850/1676] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:18:59 2022] POST /identity/v3/users => generated 282 bytes in 253 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:18:59.504783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.520114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3c4aa5-7c7d-4e88-9225-e68b482f22f3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.538078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.538781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.539043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.539333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.540052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.541863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.562530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 827/1677] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:59.563782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.578334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243643e5-5990-446a-8e71-803687ba0683 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.591665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.592353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.592635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.592907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.593570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.596982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ca3aa7b-5142-47e6-b6e6-02a516b1d9a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.629632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:18:59.630099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 851/1678] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:18:59 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:18:59.643651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4485729-2eb5-4ca4-a506-f1ed66cb0a83 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:18:59.653788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:18:59.654663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.654777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.655100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] PATH_INFO: `/v3/projects/705cd438493847f5b80570aad8f64915/users/e453bb6973ec418dab23d48cb9182275/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.693470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=705cd438493847f5b80570aad8f64915, user_id=e453bb6973ec418dab23d48cb9182275, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:18:59.697451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:18:59.729000 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 828/1679] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:18:59 2022] PUT /identity/v3/projects/705cd438493847f5b80570aad8f64915/users/e453bb6973ec418dab23d48cb9182275/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:18:59.740291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8696771-7e7e-407d-99a9-75138517649e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:18:59.740823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8696771-7e7e-407d-99a9-75138517649e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:18:59.741246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8696771-7e7e-407d-99a9-75138517649e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:18:59.804645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d8696771-7e7e-407d-99a9-75138517649e None None] MFA Rules not processed for user `e453bb6973ec418dab23d48cb9182275`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:18:59.806292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d8696771-7e7e-407d-99a9-75138517649e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:18:59.969870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 852/1680] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:18:59 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:18:59.970366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.115893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da928f68-68ee-4108-81b9-f0b86de08d9c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.131608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.132416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.132704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.132996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.133626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.135677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.213054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 829/1681] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:18:59 2022] POST /identity/v3/projects => generated 365 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:00.218116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8696771-7e7e-407d-99a9-75138517649e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.239352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8696771-7e7e-407d-99a9-75138517649e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.407051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.407877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.408227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.408614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.409405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.419394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.472048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 853/1682] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:00 2022] POST /identity/v3/users => generated 289 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:00.476158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.491894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac19dfbb-fdf8-475c-92ac-d9fc1be3a953 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.503079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.504077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.504783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.505281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.506287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.508237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.535749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 830/1683] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:00.537851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.551294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b25f911-6be5-410e-97f8-cdad2ffa745d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.561382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.562662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.563194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.566076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] PATH_INFO: `/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5/users/039e8111b4984b1a9b8326a6358f6000/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.605640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337896f2ae1a4f9ca0aea1a665ffc6f5, user_id=039e8111b4984b1a9b8326a6358f6000, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.613415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.648283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 854/1684] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:00 2022] PUT /identity/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5/users/039e8111b4984b1a9b8326a6358f6000/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:00.653174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.667410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8293bd9d-d786-459e-945d-217205493eaf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.750074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:00.815806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.816438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.816714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.817036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.817692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.822099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2981de26-947c-45a7-89ec-ea67945881a8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.853792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 831/1685] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:00 2022] GET /identity/v3/roles => generated 1420 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:00.856054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:00.871027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ca00de-c414-40f3-9dba-346c97672b00 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:00.884914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:00.885717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.886001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.886483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] PATH_INFO: `/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5/users/039e8111b4984b1a9b8326a6358f6000/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:00.891696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=337896f2ae1a4f9ca0aea1a665ffc6f5, user_id=039e8111b4984b1a9b8326a6358f6000, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:00.899826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:00.916727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:00.938096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 855/1686] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:00 2022] PUT /identity/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5/users/039e8111b4984b1a9b8326a6358f6000/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:00.948883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:00.949183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:00.949460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:01.011451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] MFA Rules not processed for user `039e8111b4984b1a9b8326a6358f6000`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:01.016688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:01.209806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 832/1687] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:00 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:01.302075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:01.305010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:01.441432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd01215-785f-4258-bf13-5f58c7db57d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:01.447016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:01.447938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:01.448393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:01.448836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:01.465567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:01.468739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb8a2cc-55c3-4683-a71c-885e24ee0dc4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:01.489512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 856/1688] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:01 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:01.767573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:01.770434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:01.834044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886cf999-f391-4816-8b03-26683aaa87bc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:01.841287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:01.841964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:01.842363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:01.842552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:02.015626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:02.020694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:02.033908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 833/1689] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:02.105319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe4d38-2140-4776-aa73-890555436aac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:02.105611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe4d38-2140-4776-aa73-890555436aac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:02.105863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafe4d38-2140-4776-aa73-890555436aac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:02.144350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eafe4d38-2140-4776-aa73-890555436aac None None] MFA Rules not processed for user `96d18f6efa1f4c91a6f213c095c507a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:02.145808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eafe4d38-2140-4776-aa73-890555436aac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:02.316847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 857/1690] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:02 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:02.335223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:02.344395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:02.442867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:02.499413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e75a738-f93e-44c4-b071-30b2abb5262f service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:02.513575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e456-b1ce-496e-b1f0-00d5d4baf0ed None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:02.513831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e456-b1ce-496e-b1f0-00d5d4baf0ed None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:02.514107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7819e456-b1ce-496e-b1f0-00d5d4baf0ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:02.523261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:02.523944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:02.526869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:02.527156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:02.561949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:02.567873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1a05f-54ce-4d60-bc00-c8f1c30928d1 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:02.602872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 834/1691] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 268 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:02.609262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7819e456-b1ce-496e-b1f0-00d5d4baf0ed None None] MFA Rules not processed for user `9c354bbd0e7a414795f4e349f57d6a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:02.611304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7819e456-b1ce-496e-b1f0-00d5d4baf0ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:02.863137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 858/1692] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:02 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 352 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:03.368904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:03.369407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:03.369483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:03.505849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] MFA Rules not processed for user `eb61948f41344ea481024a96cf03db37`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:03.507128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:03.770447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 835/1693] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 403 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:03.781552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3be79-b66e-42c6-97ef-b83c7488a3c9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:03.783252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3be79-b66e-42c6-97ef-b83c7488a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:03.783252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3be79-b66e-42c6-97ef-b83c7488a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:03.815264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8be3be79-b66e-42c6-97ef-b83c7488a3c9 None None] MFA Rules not processed for user `eb61948f41344ea481024a96cf03db37`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:03.820789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8be3be79-b66e-42c6-97ef-b83c7488a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:03.859454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:03.860078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 859/1694] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:03 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:03.862166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:03.913101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beabce77-9cee-41a5-a1de-815aa14a1ec7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:03.934617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:03.935259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:03.935541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:03.937300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:03.967341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:03.970225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a27690-2d38-4af3-8541-76237db6cefc service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:04.012522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 836/1695] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:06.894627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27a83-0294-4a4c-a797-f00aa6881eeb None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:06.895264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27a83-0294-4a4c-a797-f00aa6881eeb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:06.895264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c27a83-0294-4a4c-a797-f00aa6881eeb None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:06.896827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 860/1696] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 30 04:19:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:06.908315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85578a6-8d87-41e2-aa41-a95ad813123f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:06.908813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85578a6-8d87-41e2-aa41-a95ad813123f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:06.908894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85578a6-8d87-41e2-aa41-a95ad813123f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:07.034580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e85578a6-8d87-41e2-aa41-a95ad813123f None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:07.035964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e85578a6-8d87-41e2-aa41-a95ad813123f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:07.177427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 837/1697] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:19:06 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:07.299785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:07.300074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:07.300353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:07.301719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 861/1698] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:19:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Aug 30 04:19:07.349112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:07.351064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:07.353787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f35dbd-e42f-4f52-83f6-4beb21c3da21 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:07.354845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f35dbd-e42f-4f52-83f6-4beb21c3da21 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:07.355850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f35dbd-e42f-4f52-83f6-4beb21c3da21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:07.379183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3dc15b6-69c0-4e6b-9568-7bb148537d37 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:07.400342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:07.401773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:07.402052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:07.402342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:07.424302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b7f35dbd-e42f-4f52-83f6-4beb21c3da21 None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:07.433129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b7f35dbd-e42f-4f52-83f6-4beb21c3da21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:07.436974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:07.439586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:07.451171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:07.481160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 862/1699] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:07.611065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:07.666425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 838/1700] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:19:07 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 317 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:07.763111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:07.766280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:07.791037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795bcddd-717b-4fba-acab-aeb372447d85 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:07.798572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:07.799415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:07.799848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:07.800877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:07.827131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:07.829122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:07.842682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 863/1701] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:08.084509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d78a6-bc7b-478d-97a2-f07c433dc569 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:08.085592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d78a6-bc7b-478d-97a2-f07c433dc569 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:08.086093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d78a6-bc7b-478d-97a2-f07c433dc569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:08.130740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-648d78a6-bc7b-478d-97a2-f07c433dc569 None None] MFA Rules not processed for user `8c30ec9810204a4685590c42707e41f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:08.133420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-648d78a6-bc7b-478d-97a2-f07c433dc569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:08.277066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 839/1702] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:08.302876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:08.319420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0509e3ff-909e-4119-b006-2528ac7cae66 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:08.326734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:08.328251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:08.328689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:08.329247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:08.338173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:08.340551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5781b955-579e-4a41-8a74-bfbe2ea2cb46 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:08.366327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 864/1703] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:08 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:08.483638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:08.483972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:08.484190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:08.563923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] MFA Rules not processed for user `b0e68c885560485ead1c61fe83fc3ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:08.565373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:08.710922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 840/1704] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 228 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:08.723871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:08.724223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:08.724502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:08.793740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] MFA Rules not processed for user `b0e68c885560485ead1c61fe83fc3ce4`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:08.796351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:08.835404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 865/1705] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:08 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:08.847025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:08.884397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255509a4-9084-4263-9887-ec8ab7160481 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:08.921356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:08.925251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:08.928622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:08.930835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.012106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:09.023430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02e1b995-5f50-420e-b1a7-2034de601eaf service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:09.044456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 841/1706] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:08 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:09.115621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:09.162175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bc681f-7726-47a7-bd9a-c3f73ab6f225 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:09.177624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:09.179503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.179797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.180083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.186012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.215812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:09.218391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0817b9be-d112-46fd-a89f-d3befec6516f service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:09.248112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 866/1707] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:09.266292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d084-fa66-4bcc-91e0-ac0c4eb79f2c None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.266853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d084-fa66-4bcc-91e0-ac0c4eb79f2c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.267789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618d084-fa66-4bcc-91e0-ac0c4eb79f2c None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.270479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 842/1708] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:19:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:09.389223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172d495-a50d-42d9-8dc4-7e1706b0052d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.390422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172d495-a50d-42d9-8dc4-7e1706b0052d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.390848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0172d495-a50d-42d9-8dc4-7e1706b0052d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.451914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0172d495-a50d-42d9-8dc4-7e1706b0052d None None] MFA Rules not processed for user `eb61948f41344ea481024a96cf03db37`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:09.456199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0172d495-a50d-42d9-8dc4-7e1706b0052d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.486781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 867/1709] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:09 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:09.551979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f244-0461-4959-9a03-fb1b8d33faff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.552267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f244-0461-4959-9a03-fb1b8d33faff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.552545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3842f244-0461-4959-9a03-fb1b8d33faff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.590452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3842f244-0461-4959-9a03-fb1b8d33faff None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:09.596596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3842f244-0461-4959-9a03-fb1b8d33faff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.629564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 843/1710] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:19:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:09.727202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f6b8-fb51-426f-83ac-f5d75ae107b9 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.727610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f6b8-fb51-426f-83ac-f5d75ae107b9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.727892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7751f6b8-fb51-426f-83ac-f5d75ae107b9 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.730304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 868/1711] 10.222.0.46 () {54 vars in 961 bytes} [Tue Aug 30 04:19:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:09.736342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.737176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.737618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.784514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.784804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.787116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.804058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] MFA Rules not processed for user `ba39d57469d34f258aa903bff3935c3f`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:09.808012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.839325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 844/1712] 10.222.0.46 () {58 vars in 1058 bytes} [Tue Aug 30 04:19:09 2022] POST /identity/v3/auth/tokens => generated 3308 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:09.840580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] MFA Rules not processed for user `96d18f6efa1f4c91a6f213c095c507a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:09.843678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.847374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:09.871439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 869/1713] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:09.873235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b14bcec-01e4-4900-a523-d4c95578556b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:09.888711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:09.889993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:09.890490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:09.891493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:09.893173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:09.896125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:09.933763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:09.935711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d3987-3f90-4340-a460-e056b4424a78 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:09.957261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 845/1714] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:09 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:10.058599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba8f29fa-0afa-495b-a5a0-c8774d88b195 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:10.068951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:10.069651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:10.069934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:10.073558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] PATH_INFO: `/v3/users/47f52ec04d454dff861474eb9eb09d5f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:10.075233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] RBAC: Authorizing `identity:delete_user(user_id=47f52ec04d454dff861474eb9eb09d5f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:10.079356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:10.190676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47f52ec04d454dff861474eb9eb09d5f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:10.220569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47f52ec04d454dff861474eb9eb09d5f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:10.223604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47f52ec04d454dff861474eb9eb09d5f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:10.224737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47f52ec04d454dff861474eb9eb09d5f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:10.245725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-16d8c6fa-61c1-4e33-be15-402f406f30ad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47f52ec04d454dff861474eb9eb09d5f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:10.273965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 870/1715] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:09 2022] DELETE /identity/v3/users/47f52ec04d454dff861474eb9eb09d5f => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:10.420321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:10.420321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:10.424121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:10.429517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 846/1716] 10.222.0.46 () {54 vars in 912 bytes} [Tue Aug 30 04:19:10 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:10.469217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:10.469217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:10.470131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:10.551991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:10.563592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:10.593161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:10.695324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 871/1717] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:19:10 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:10.737546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:10.739915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:10.769295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a4c0fe-c970-4841-bd1d-432f16eaaa33 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:10.774376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:10.775341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:10.775931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:10.776523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:10.796653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:10.800724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:10.837862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 847/1718] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:10 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:11.709635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:11.711162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:11.933904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f430ed-01c5-4793-a2a3-8cd84618c132 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:11.946908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:11.948579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:11.949064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:11.950994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] PATH_INFO: `/v3/projects/8be1dd6cbfc544c4b430b2420bb6bdca` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:11.952475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] RBAC: Authorizing `identity:delete_project(project_id=8be1dd6cbfc544c4b430b2420bb6bdca)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:11.953760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: WARNING py.warnings [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Aug 30 04:19:11.953760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: warnings.warn(msg) Aug 30 04:19:11.953760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: Aug 30 04:19:11.980369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:12.157113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8be1dd6cbfc544c4b430b2420bb6bdca', 'request_id': 'req-aa90432d-be26-4137-98c3-1123393debba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.209257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8be1dd6cbfc544c4b430b2420bb6bdca', 'request_id': 'req-aa90432d-be26-4137-98c3-1123393debba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.236471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8be1dd6cbfc544c4b430b2420bb6bdca', 'request_id': 'req-aa90432d-be26-4137-98c3-1123393debba'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.314243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] The token cache is being invalidate because project 8be1dd6cbfc544c4b430b2420bb6bdca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:12.314951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.317952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa90432d-be26-4137-98c3-1123393debba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.320787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 872/1719] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:11 2022] DELETE /identity/v3/projects/8be1dd6cbfc544c4b430b2420bb6bdca => generated 0 bytes in 612 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:12.326076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:12.364326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51d987a0-e352-414b-85a6-4fb147286db2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:12.424572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:12.608636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:12.609005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:12.609284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:12.609542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] PATH_INFO: `/v3/users/9c354bbd0e7a414795f4e349f57d6a9a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:12.611220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9c354bbd0e7a414795f4e349f57d6a9a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:12.616127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:12.770823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9c354bbd0e7a414795f4e349f57d6a9a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.786349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9c354bbd0e7a414795f4e349f57d6a9a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.836020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9c354bbd0e7a414795f4e349f57d6a9a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.838148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9c354bbd0e7a414795f4e349f57d6a9a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.956853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9c354bbd0e7a414795f4e349f57d6a9a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:12.996468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 848/1720] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:12 2022] DELETE /identity/v3/users/9c354bbd0e7a414795f4e349f57d6a9a => generated 0 bytes in 667 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:14.123393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b38d-43cd-428c-b148-fa3ec141c77f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:14.123999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b38d-43cd-428c-b148-fa3ec141c77f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:14.124139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f09b38d-43cd-428c-b148-fa3ec141c77f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:14.396951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8f09b38d-43cd-428c-b148-fa3ec141c77f None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:14.398224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8f09b38d-43cd-428c-b148-fa3ec141c77f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:14.503075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:14.505498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:14.563254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:14.690019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 873/1721] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:19:14 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 568 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:14.729485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea356a2-fe08-4e76-8597-89c9314d2258 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:14.742774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:14.746872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:14.747578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:14.748612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] PATH_INFO: `/v3/users/98e69f7641f8431889369d108326b742` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:14.754730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] RBAC: Authorizing `identity:delete_user(user_id=98e69f7641f8431889369d108326b742)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:14.756959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:14.880159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:14.914440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98e69f7641f8431889369d108326b742'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:14.927891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98e69f7641f8431889369d108326b742'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:14.940220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98e69f7641f8431889369d108326b742'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:14.941353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98e69f7641f8431889369d108326b742'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:14.974969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98e69f7641f8431889369d108326b742'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:14.994269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 849/1722] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:14 2022] DELETE /identity/v3/users/98e69f7641f8431889369d108326b742 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:14.996212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:14.996560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:14.996839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:15.011256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:15.033199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.067544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:15.068788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.253774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 874/1723] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:14 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:15.255963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:15.271481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.313504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26af717b-df35-4721-805a-153eb09c6b09 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:15.321319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:15.322165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:15.322650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:15.323754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] PATH_INFO: `/v3/projects/d7f48b5d5da647cba4bf5d0622348383` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:15.326175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7f48b5d5da647cba4bf5d0622348383)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:15.329105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:15.386235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da4f480-e0b1-40d1-9ba9-cbcf8e3dfb6a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:15.401481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:15.409244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:15.409553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:15.409837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:15.412019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.447105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7f48b5d5da647cba4bf5d0622348383', 'request_id': 'req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:15.483710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7f48b5d5da647cba4bf5d0622348383', 'request_id': 'req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:15.509628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7f48b5d5da647cba4bf5d0622348383', 'request_id': 'req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:15.559390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] The token cache is being invalidate because project d7f48b5d5da647cba4bf5d0622348383 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:15.559759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:15.564775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b34d15f8-dbb8-4402-83bc-295d2c50e7cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:15.576887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 875/1724] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:15 2022] DELETE /identity/v3/projects/d7f48b5d5da647cba4bf5d0622348383 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:15.578269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c148-2d8f-44a3-940f-377cb5cf29ee None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:15.578781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c148-2d8f-44a3-940f-377cb5cf29ee None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:15.578903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a60c148-2d8f-44a3-940f-377cb5cf29ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:15.644131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a60c148-2d8f-44a3-940f-377cb5cf29ee None None] MFA Rules not processed for user `8c30ec9810204a4685590c42707e41f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:15.647351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a60c148-2d8f-44a3-940f-377cb5cf29ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.716750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:15.722766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:15.734438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:15.749024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 850/1725] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:19:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 743 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:19:15.752461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:15.756561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:15.861346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 876/1726] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:15 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:15.875357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:15.878089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18410-c062-44f0-a220-4e976d27035a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:15.882025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18410-c062-44f0-a220-4e976d27035a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:15.883787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca18410-c062-44f0-a220-4e976d27035a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:15.970134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6793d3a-11d2-4577-8a76-d861f50ac262 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:16.045802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eca18410-c062-44f0-a220-4e976d27035a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:16.061634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eca18410-c062-44f0-a220-4e976d27035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:16.155090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:16.156288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.156904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.157280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.159376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:16.414895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:16.421301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c2ec69-eced-4f8f-8eb2-6fe97a9feaec service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:16.446624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 877/1727] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 585 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:16.452771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18410-c062-44f0-a220-4e976d27035a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:16.461699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eca18410-c062-44f0-a220-4e976d27035a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:16.478352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 851/1728] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 728 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:16.486842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.487687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.487987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.514966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca18410-c062-44f0-a220-4e976d27035a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:16.569580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:16.574379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.575033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.575755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] PATH_INFO: `/v3/users/e453bb6973ec418dab23d48cb9182275` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.583492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e453bb6973ec418dab23d48cb9182275)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:16.595070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:16.615453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:16.616665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:16.649781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 852/1729] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:16.659001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:16.680812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4d73f-a8aa-4338-b6df-9dcc7c94fd54 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:16.696927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:16.697808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.698285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.699662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.700366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:16.701328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:16.704149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:16.759500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 853/1730] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:16.762832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e453bb6973ec418dab23d48cb9182275'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:16.764729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:16.793715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903c7196-b148-45d4-9c09-f39e50441483 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:16.811475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e453bb6973ec418dab23d48cb9182275'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:16.814472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e453bb6973ec418dab23d48cb9182275'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:16.816252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e453bb6973ec418dab23d48cb9182275'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:16.826734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:16.834765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.835095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.835396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.843187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:16.849704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:16.861512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0d32d76d-b437-4455-a62f-ec67ca5812d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e453bb6973ec418dab23d48cb9182275'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:16.907235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 878/1731] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:16 2022] DELETE /identity/v3/users/e453bb6973ec418dab23d48cb9182275 => generated 0 bytes in 457 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:16.911919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:16.912252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:16.912488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:16.918744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 854/1732] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:16 2022] POST /identity/v3/projects => generated 383 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:16.923057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:16.929351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:16.999984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:17.001748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:17.117376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:17.154790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a83e3c1b-2de0-42ea-ab49-7dfa80b8d9ef admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:17.174450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:17.175806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.176103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.177335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 879/1733] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:16 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:17.178571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.183405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:17.190197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:17.293449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:17.294285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 855/1734] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:16 2022] POST /identity/v3/users => generated 292 bytes in 372 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:17.317826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b691250e-8982-4bbb-8c33-d108a0cf97aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:17.332979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:17.333615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.336824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.337241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.340260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:17.341920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:17.348790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:17.350575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:17.367168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:17.390120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 880/1735] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:17.391603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:17.415261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80759bfb-ca30-4ccb-afc1-3c055381677b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:17.429638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:17.434680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.436375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.436806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.440727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:17.444611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:17.477155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 881/1736] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:17 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:17.484397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:17.514313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c673a7-0e74-4067-b865-89f9f377caad admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:17.521241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:17.522179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.522378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.522648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] PATH_INFO: `/v3/projects/9a1fb85fdfb040f48c82d62196bbc21d/users/640d00f2ed09464b8a69fcee5ce74988/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.537797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc179b86-7d4c-4fc1-a022-b8c08fa531f1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:17.542866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:17.543838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.544293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.544570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.559258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:17.560871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e38e566-3eb2-45d8-aba9-5ac8fd3c6875 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:17.563793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1fb85fdfb040f48c82d62196bbc21d, user_id=640d00f2ed09464b8a69fcee5ce74988, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:17.567060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:17.575793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 856/1737] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:17 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 228 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:17.595678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 882/1738] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:17 2022] PUT /identity/v3/projects/9a1fb85fdfb040f48c82d62196bbc21d/users/640d00f2ed09464b8a69fcee5ce74988/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:17.620546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:17.620546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:17.620546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:17.725027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] MFA Rules not processed for user `640d00f2ed09464b8a69fcee5ce74988`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:17.731640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:18.020249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 857/1739] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 409 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:18.027581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:18.229019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566619c-f105-4eff-8819-8ed2b5190def admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:18.241696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:18.242833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:18.243272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:18.243711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:18.245174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:18.247823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:18.335438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 883/1740] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:18 2022] POST /identity/v3/projects => generated 383 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:18.341241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:18.365082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7011fcc-dde9-4c11-9340-85e55cc9490e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:18.559591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:18.561089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:18.561376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:18.561673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:18.564443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:18.570007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:18.654007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 858/1741] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:18 2022] POST /identity/v3/users => generated 298 bytes in 313 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:18.658751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:18.679432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e53435e-f4b0-4bfb-bd6f-19e0689d2b83 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:18.722455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:18.723232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:18.723519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:18.726355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:18.727756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:18.735834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:18.783555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 884/1742] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:18.788416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:18.815351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9960bc99-03f1-483c-b406-cab57c66fd9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:18.836537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:18.841030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:18.841391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:18.842432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] PATH_INFO: `/v3/projects/8cdf00d9344943108f6407cac08d5f1a/users/dce62dc46d9b45c9b17198ffc40186a3/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:18.859932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:18.914740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdf00d9344943108f6407cac08d5f1a, user_id=dce62dc46d9b45c9b17198ffc40186a3, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:18.922926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:18.957245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 859/1743] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:18 2022] PUT /identity/v3/projects/8cdf00d9344943108f6407cac08d5f1a/users/dce62dc46d9b45c9b17198ffc40186a3/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:18.962330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:18.977647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57601f06-65a8-44ac-bcad-2f38d3b3a092 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:19.092656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:19.093471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.093899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.094399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.095379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:19.097131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:19.119008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 885/1744] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:18 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:19.120174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:19.149538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb8b0e9-3cde-476f-ac86-984a6b4b992f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:19.199480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:19.200588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.200588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.204495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] PATH_INFO: `/v3/projects/8cdf00d9344943108f6407cac08d5f1a/users/dce62dc46d9b45c9b17198ffc40186a3/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.227951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8cdf00d9344943108f6407cac08d5f1a, user_id=dce62dc46d9b45c9b17198ffc40186a3, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:19.240184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fbefa-f1b2-4c56-b843-c0b9eefe44f4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:19.270044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 860/1745] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:19 2022] PUT /identity/v3/projects/8cdf00d9344943108f6407cac08d5f1a/users/dce62dc46d9b45c9b17198ffc40186a3/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:19.283863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:19.284635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934775-546e-4490-91b2-52217455bffd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.285588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934775-546e-4490-91b2-52217455bffd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.285990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:19.286378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a934775-546e-4490-91b2-52217455bffd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.302288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af8e02d-a24b-4ed3-8509-1545e0b6ec5b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:19.356861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8a934775-546e-4490-91b2-52217455bffd None None] MFA Rules not processed for user `dce62dc46d9b45c9b17198ffc40186a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:19.358977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a934775-546e-4490-91b2-52217455bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:19.370604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:19.407256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:19.408167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.408673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.409180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] PATH_INFO: `/v3/projects/a9569ed0183d40ebbc08c543421178be` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.410574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9569ed0183d40ebbc08c543421178be)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:19.413041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:19.496372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9569ed0183d40ebbc08c543421178be', 'request_id': 'req-5f58e05d-e776-4ad3-af29-052c757e2a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:19.520481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9569ed0183d40ebbc08c543421178be', 'request_id': 'req-5f58e05d-e776-4ad3-af29-052c757e2a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:19.548476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9569ed0183d40ebbc08c543421178be', 'request_id': 'req-5f58e05d-e776-4ad3-af29-052c757e2a86'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:19.583383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] The token cache is being invalidate because project a9569ed0183d40ebbc08c543421178be was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:19.583834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:19.584628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5f58e05d-e776-4ad3-af29-052c757e2a86 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:19.586748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 886/1746] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:19 2022] DELETE /identity/v3/projects/a9569ed0183d40ebbc08c543421178be => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:19.593177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef809b-60fc-454c-a620-9068596a14d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.593502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef809b-60fc-454c-a620-9068596a14d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.593901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ef809b-60fc-454c-a620-9068596a14d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.626712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 861/1747] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:19 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:19.640774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934775-546e-4490-91b2-52217455bffd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:19.643669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8a934775-546e-4490-91b2-52217455bffd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:19.654489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d7ef809b-60fc-454c-a620-9068596a14d5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:19.655843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d7ef809b-60fc-454c-a620-9068596a14d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:19.861808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a934775-546e-4490-91b2-52217455bffd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:19.923154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 887/1748] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:19.931492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:19.931877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:19.932781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:19.979188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:19.980446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:20.015778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 888/1749] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:20.020388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:20.022129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:20.023053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.023514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.023965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.031244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:20.042164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42224404-514d-47e1-9b21-60e56a9d8fa5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:20.060649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:20.061270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.061548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.061816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.062503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:20.062802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:20.064622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:20.104581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:20.108735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 889/1750] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:19:20.116188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:20.147771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b7752e3-157d-481d-8b69-3c302f2cacf5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:20.164444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:20.165008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.171060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.171441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.177942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:20.180106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:20.199692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:20.201372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:20.235452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 862/1751] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:19 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:20.303132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 890/1752] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:20 2022] POST /identity/v3/projects => generated 401 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:20.307829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:20.328140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05dbf3a9-b277-4713-931e-c91fc93f62e1 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:20.461336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:20.463596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:20.537767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:20.559504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:20.562452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.563396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.563803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.565196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:20.573461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2749342-b43b-452e-bbdc-519764455bd4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:20.640012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c155a902-a683-44f1-b356-9c454cd20761 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:20.648709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 863/1753] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:20 2022] POST /identity/v3/users => generated 301 bytes in 339 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:20.649437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.657607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.659239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.672502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:20.677015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:20.677529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:20.677951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:20.694164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:20.773261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:20.777694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:20.992139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:20.993591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:21.007154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 864/1754] 10.222.0.75 () {58 vars in 1015 bytes} [Tue Aug 30 04:19:20 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 360 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:21.008682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:21.037638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc4d4eee-cb84-4ca2-aaaf-11ff850d0608 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:21.099743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 891/1755] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 611 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:21.100111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:21.100111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:21.100111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:21.100111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:21.100404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:21.100475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:21.100475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:21.116707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:21.167771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d79405e-1ba4-4da3-b1e3-51e3f6465876 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:21.186010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 865/1756] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:21.186736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:21.187276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:21.187559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:21.187838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:21.191683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:21.223043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:21.229679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8813ac-daa0-458a-bcc3-f0e4c92bfc47 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:21.232543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:21.259089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:21.259725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:21.260102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:21.260699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:21.277639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:21.279458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a38d03-e54c-4e99-a4b2-b934fec14708 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:21.281813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 892/1757] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:21 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:21.318337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:21.332052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 866/1758] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:21 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:21.346041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf321fc-e218-441b-960d-04559a46b4ef service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:21.375176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:21.375878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:21.376157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:21.376456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] PATH_INFO: `/v3/projects/5e8ca740f70a4892b84a5b0abbb0ea44/users/e2bf76638be84d17ac0695aa185e67f3/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:21.449061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] RBAC: Authorizing `identity:create_grant(project_id=5e8ca740f70a4892b84a5b0abbb0ea44, user_id=e2bf76638be84d17ac0695aa185e67f3, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:21.458692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:21.512380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 893/1759] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:21 2022] PUT /identity/v3/projects/5e8ca740f70a4892b84a5b0abbb0ea44/users/e2bf76638be84d17ac0695aa185e67f3/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:21.513870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:21.513870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:21.514088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:21.636622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] MFA Rules not processed for user `e2bf76638be84d17ac0695aa185e67f3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:21.649275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:21.777719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:21.907060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 867/1760] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:21 2022] POST /identity/v3/auth/tokens => generated 3278 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:21.912297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.146173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62445ad3-98c6-4bce-8d79-5e5ba08c9caf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.163754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.165650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.165944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.169978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.171403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.176537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.245740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 894/1761] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:21 2022] POST /identity/v3/projects => generated 401 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:22.248003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.273422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e80785-dfc0-4c97-bd4d-0d5ee84aba6b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.462050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.464466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:22.486567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.487194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.487475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.490308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.491010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.492983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.499493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec468b49-79a4-4347-a2be-360831f46f31 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.511188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.511851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.512133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.512462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] PATH_INFO: `/v3/projects/705cd438493847f5b80570aad8f64915` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.514873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=705cd438493847f5b80570aad8f64915)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.519508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.539657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 868/1762] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:22 2022] POST /identity/v3/users => generated 307 bytes in 292 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:22.545474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.564131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26c34cd-0bf2-40ce-bdd8-9082c68aee58 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.575818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.576541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.576713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.576977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.581786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.583657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.594886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:22.603143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 869/1763] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:22.609052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.630065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '705cd438493847f5b80570aad8f64915', 'request_id': 'req-eab4ddb1-e985-40db-a87b-069bc24372b5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:22.637498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51577b5d-3120-4b54-8257-4ba36df1bc1a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.640814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '705cd438493847f5b80570aad8f64915', 'request_id': 'req-eab4ddb1-e985-40db-a87b-069bc24372b5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:22.654677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '705cd438493847f5b80570aad8f64915', 'request_id': 'req-eab4ddb1-e985-40db-a87b-069bc24372b5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:22.676301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] The token cache is being invalidate because project 705cd438493847f5b80570aad8f64915 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:22.677311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:22.683273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:22.685159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 895/1764] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:22 2022] DELETE /identity/v3/projects/705cd438493847f5b80570aad8f64915 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:22.689086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.692135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:22.773988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.773988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.773988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.773988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] PATH_INFO: `/v3/projects/eace01176d9143c0a754ea9ebed6d34a/users/58a853c4922a47fba91ac5bbb1b7e668/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.841063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eace01176d9143c0a754ea9ebed6d34a, user_id=58a853c4922a47fba91ac5bbb1b7e668, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.843976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.844820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab4ddb1-e985-40db-a87b-069bc24372b5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:22.859139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:22.860112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:22.860243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:22.860641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] PATH_INFO: `/v3/users/039e8111b4984b1a9b8326a6358f6000` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:22.861971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=039e8111b4984b1a9b8326a6358f6000)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:22.864529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:22.896231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 870/1765] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:22 2022] PUT /identity/v3/projects/eace01176d9143c0a754ea9ebed6d34a/users/58a853c4922a47fba91ac5bbb1b7e668/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:22.900205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:22.903660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.010062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '039e8111b4984b1a9b8326a6358f6000'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.024149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:23.039407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '039e8111b4984b1a9b8326a6358f6000'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.040247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '039e8111b4984b1a9b8326a6358f6000'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.041060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '039e8111b4984b1a9b8326a6358f6000'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.055248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '039e8111b4984b1a9b8326a6358f6000'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.125148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 896/1766] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:22 2022] DELETE /identity/v3/users/039e8111b4984b1a9b8326a6358f6000 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:23.135315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a36f9-8b4a-43fe-9b10-1af78913ca32 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:23.251584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:23.253269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.253700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.254243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.255207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:23.257080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43808401-44e2-4df9-918f-479f66791f13 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:23.274608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 871/1767] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:22 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:23.282121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:23.291745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.318574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e641a2-66d2-43ee-afc7-2a8cf4bb4ea4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:23.334748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:23.335436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.335842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.336126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] PATH_INFO: `/v3/projects/eace01176d9143c0a754ea9ebed6d34a/users/58a853c4922a47fba91ac5bbb1b7e668/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.338295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eace01176d9143c0a754ea9ebed6d34a, user_id=58a853c4922a47fba91ac5bbb1b7e668, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:23.340941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:23.373641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 897/1768] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:23 2022] PUT /identity/v3/projects/eace01176d9143c0a754ea9ebed6d34a/users/58a853c4922a47fba91ac5bbb1b7e668/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 95 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:23.384834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffbc2e-b89f-40cb-9330-c2c2c95ef188 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.386049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffbc2e-b89f-40cb-9330-c2c2c95ef188 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.386357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffbc2e-b89f-40cb-9330-c2c2c95ef188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.430497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-beffbc2e-b89f-40cb-9330-c2c2c95ef188 None None] MFA Rules not processed for user `58a853c4922a47fba91ac5bbb1b7e668`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:23.434644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-beffbc2e-b89f-40cb-9330-c2c2c95ef188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.533950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 872/1769] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3345 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:23.544904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:23.546892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.584528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=17,keystone:SELECT=80,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:23.645657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a121dd-8975-448a-9628-100c5639ffe2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:23.655828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:23.662374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.662712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.663302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.677319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.677672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.678010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.683057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 873/1770] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:19:23 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:23.689170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:23.691054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4610020e-aede-40c5-a778-27eeca90cbe0 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:23.703425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:23.705869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.726453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 898/1771] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:23 2022] GET /identity/v3/auth/tokens => generated 3345 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:23.734634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.735057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.735441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.761957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:23.764354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.863761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf0acee-0e3b-4fd6-a43d-f754ac8bef82 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:23.876046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:23.877250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.877583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.877801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] PATH_INFO: `/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.880671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] RBAC: Authorizing `identity:delete_project(project_id=337896f2ae1a4f9ca0aea1a665ffc6f5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:23.882635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:23.884640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:23.922070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 899/1772] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:19:23 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:23.922975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:23.924273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.940324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c543e5b-ab39-4a7f-9992-61ccf82e7c2c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:23.944396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:23.945026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:23.945308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:23.945582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:23.947557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:23.981887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '337896f2ae1a4f9ca0aea1a665ffc6f5', 'request_id': 'req-a0b13f66-69da-4ec5-9ff1-a467d2242477'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:23.992098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '337896f2ae1a4f9ca0aea1a665ffc6f5', 'request_id': 'req-a0b13f66-69da-4ec5-9ff1-a467d2242477'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:24.001503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '337896f2ae1a4f9ca0aea1a665ffc6f5', 'request_id': 'req-a0b13f66-69da-4ec5-9ff1-a467d2242477'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:24.014429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] The token cache is being invalidate because project 337896f2ae1a4f9ca0aea1a665ffc6f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:24.014867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:24.015638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:24.017620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 874/1773] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:23 2022] DELETE /identity/v3/projects/337896f2ae1a4f9ca0aea1a665ffc6f5 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:24.018535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.021869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:24.032123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.033796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b4a88b3-a917-4db0-a926-f53ca9c96e26 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.045482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 900/1774] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:23 2022] GET /identity/v3/auth/tokens => generated 3278 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:24.047885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.048350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.048793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.081424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:24.082669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:24.108582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b13f66-69da-4ec5-9ff1-a467d2242477 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.114386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.114995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.115272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.115536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.117558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:24.132883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.133450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:24.135865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c25b9b-72ba-4671-855e-7ab1536a850b service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.152645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 875/1775] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:24 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:24.158307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 901/1776] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:24.164944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.165237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.165484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.187292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:24.188557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:24.198288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 876/1777] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:24.202377 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.218459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e8232a-e23b-4d15-ada0-230e0f80ad4e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.224367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.225225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.225497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.225802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.226778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:24.227024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.228605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.241852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 902/1778] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:24.246773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.256107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b2b64f-3c00-4266-a46d-4836f270b820 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.261701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.262349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.262626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.262893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.263541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.265441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.297451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 877/1779] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/projects => generated 367 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:24.299396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.311112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f7337-5656-4dc5-b94f-03b558ba0a1f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.385217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.386050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.386420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.386705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.387476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.389427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.412219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 903/1780] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/users => generated 284 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:24.416921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.428643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15da14fa-1d82-47cb-8f25-4bcbdd6dc372 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.434849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.435534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.435799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.436082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.436782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.438475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.451081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 878/1781] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:24.455206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.465336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df10ad21-3e48-455b-aefa-dd94ba269fe6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.469934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.470556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.470834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.471110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.471728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.473302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb93b214-d82d-49d8-be57-c3ae0f6aee9b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.484644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 904/1782] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:24 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:24.489011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.499807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0dca62-319e-4387-890d-ebc9bdedeac1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.504447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.505085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.505367 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.505640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] PATH_INFO: `/v3/projects/c60fbd06968e49b7a160539eeb9e0e09/users/1d21e8e55dc5448faba251fc9a1fd816/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.520604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c60fbd06968e49b7a160539eeb9e0e09, user_id=1d21e8e55dc5448faba251fc9a1fd816, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.523291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.536127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 879/1783] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:24 2022] PUT /identity/v3/projects/c60fbd06968e49b7a160539eeb9e0e09/users/1d21e8e55dc5448faba251fc9a1fd816/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:24.547206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.547497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.547765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.583139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] MFA Rules not processed for user `1d21e8e55dc5448faba251fc9a1fd816`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:24.584435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:24.668665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 905/1784] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:24.669660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.695658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:24.790770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87c1487-a295-405b-87ec-a7f281e681d2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.801031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:24.801671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:24.801954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:24.802283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:24.803050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:24.805013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:24.852145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 880/1785] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/projects => generated 369 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:24.857156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:24.890643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c51bdb2-83e7-4797-94c0-c6d8d0b6b860 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:24.965799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:25.001925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.002762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.003055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.003325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.004058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.006106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.039260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 906/1786] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:24 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:25.044267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:25.055915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec76722-7f9e-4f79-bd49-d1f4fb4c1cb9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:25.060872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.061486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.061767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.062038 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.062726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.064375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.085329 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 881/1787] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:25.089523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:25.118077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1f8373-7ab1-4b9f-b4f9-9ff1c1b93198 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:25.124824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.132166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.132460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.132800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] PATH_INFO: `/v3/projects/3eb3ad7255064fbea7968e327310fc0c/users/5aea789eecdd40538f4fce67c110e943/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.179835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb3ad7255064fbea7968e327310fc0c, user_id=5aea789eecdd40538f4fce67c110e943, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.182871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.221233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 907/1788] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:25 2022] PUT /identity/v3/projects/3eb3ad7255064fbea7968e327310fc0c/users/5aea789eecdd40538f4fce67c110e943/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:25.226962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:25.247820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba4922e-50cf-4602-918b-94e914fc95cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:25.343517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.345103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.349057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.349057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.349057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.349057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a51cd1f-9f22-4402-b366-b9ae4a630de3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.400030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 882/1789] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:25 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:25.410273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:25.428112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc04fe-95d6-4b4b-9983-534a3d04f5df admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:25.442956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.443643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.443922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.444193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] PATH_INFO: `/v3/projects/3eb3ad7255064fbea7968e327310fc0c/users/5aea789eecdd40538f4fce67c110e943/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.450201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3eb3ad7255064fbea7968e327310fc0c, user_id=5aea789eecdd40538f4fce67c110e943, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.452919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.482309 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 908/1790] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:25 2022] PUT /identity/v3/projects/3eb3ad7255064fbea7968e327310fc0c/users/5aea789eecdd40538f4fce67c110e943/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:25.493101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.493590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.493784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.567965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] MFA Rules not processed for user `5aea789eecdd40538f4fce67c110e943`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:25.569300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:25.734129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 883/1791] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:25 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:25.851521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:25.853483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:25.949723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc5a530-9d77-471f-afad-cce4d85acf80 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:25.954291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:25.954291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:25.954598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:25.954872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:25.968404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:25.968404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:25.979093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 909/1792] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:25 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:26.076284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:26.088297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a5db69-faa5-4dfd-b725-e1b4c3b50c11 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:26.095590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:26.096555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:26.097097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:26.097647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:26.110306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:26.259214 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:26.260822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23f1588-21c8-4286-81fd-84fc053e878d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:26.306059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 884/1793] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:29.253164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:29.254919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:29.331796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91895d-6d9e-4e1e-bc61-75ca63eedbc5 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:29.337067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:29.338624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:29.339218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:29.339604 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] PATH_INFO: `/v3/users/640d00f2ed09464b8a69fcee5ce74988` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:29.341068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=640d00f2ed09464b8a69fcee5ce74988)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:29.343167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:29.438019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:INSERT=5,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:29.447866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '640d00f2ed09464b8a69fcee5ce74988'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:29.471608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '640d00f2ed09464b8a69fcee5ce74988'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:29.473291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '640d00f2ed09464b8a69fcee5ce74988'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:29.474648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '640d00f2ed09464b8a69fcee5ce74988'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:29.485269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '640d00f2ed09464b8a69fcee5ce74988'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:29.505596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 910/1794] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:29 2022] DELETE /identity/v3/users/640d00f2ed09464b8a69fcee5ce74988 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:29.507739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:29.508257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:29.508577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:29.536751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:29.537884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:29.615633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 885/1795] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:29.629904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:29.631401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:29.750457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa901d2-dfde-498e-9f24-a0a9b17c01c6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:29.755848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:29.756422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:29.756695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:29.756965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:29.784403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:29.785970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ee946-f683-4b91-ba48-965c7590f22f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:29.798679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 911/1796] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:29.898155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:29.899915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:29.916582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5a175c-bf20-4fa3-bad2-57cbaf3e23a0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:29.930735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:29.931385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:29.932216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:29.932485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] PATH_INFO: `/v3/users/e2bf76638be84d17ac0695aa185e67f3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:29.935794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2bf76638be84d17ac0695aa185e67f3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:29.939508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:30.066099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2bf76638be84d17ac0695aa185e67f3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.077445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2bf76638be84d17ac0695aa185e67f3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.103288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2bf76638be84d17ac0695aa185e67f3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.105450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2bf76638be84d17ac0695aa185e67f3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.132946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2bf76638be84d17ac0695aa185e67f3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.147372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 886/1797] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:29 2022] DELETE /identity/v3/users/e2bf76638be84d17ac0695aa185e67f3 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:30.152006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.152167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:30.152279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.152557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.153771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.187513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:30.188721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:30.189946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.301682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffcc9353-e83f-4fd0-ae39-5052d3d22457 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:30.308092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:30.308681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.308967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.309244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] PATH_INFO: `/v3/users/1d21e8e55dc5448faba251fc9a1fd816` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.310568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d21e8e55dc5448faba251fc9a1fd816)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:30.312857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:30.379647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 912/1798] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:30.403206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d21e8e55dc5448faba251fc9a1fd816'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.411002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:30.412948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.433618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d21e8e55dc5448faba251fc9a1fd816'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.446932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d21e8e55dc5448faba251fc9a1fd816'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.450206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d21e8e55dc5448faba251fc9a1fd816'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.475696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-edf6a8cc-d2c9-4c78-87ff-460e616305a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d21e8e55dc5448faba251fc9a1fd816'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.486959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 887/1799] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:30 2022] DELETE /identity/v3/users/1d21e8e55dc5448faba251fc9a1fd816 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:30.491541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.492082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.492497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.505851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8095b7f1-9c33-4220-9e76-5e95ea8e41e2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:30.524784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:30.526020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:30.528123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.599681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:30.601723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.602739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.603008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.621037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.691948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 888/1800] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:30.691948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:30.693071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.708139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc4672-042a-40bb-b41a-6926296ad6b6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:30.729736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:30.731727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.732260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.732536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] PATH_INFO: `/v3/projects/9a1fb85fdfb040f48c82d62196bbc21d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.734004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1fb85fdfb040f48c82d62196bbc21d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:30.736863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:30.774321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:30.776599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:30.786467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:30.802162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 913/1801] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:30.803099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:30.805865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1fb85fdfb040f48c82d62196bbc21d', 'request_id': 'req-59502470-16ea-4e82-b9e0-fa363fd71228'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.812833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.824380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1fb85fdfb040f48c82d62196bbc21d', 'request_id': 'req-59502470-16ea-4e82-b9e0-fa363fd71228'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.828394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537960d2-e8c2-4920-88a2-308dfe952f94 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:30.843395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1fb85fdfb040f48c82d62196bbc21d', 'request_id': 'req-59502470-16ea-4e82-b9e0-fa363fd71228'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.869549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] The token cache is being invalidate because project 9a1fb85fdfb040f48c82d62196bbc21d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:30.870668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.871860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:30.874117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 889/1802] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:30 2022] DELETE /identity/v3/projects/9a1fb85fdfb040f48c82d62196bbc21d => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:30.879008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:30.882173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:30.974935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:30.975579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:30.975864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:30.976138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:30.978140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:31.011655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59502470-16ea-4e82-b9e0-fa363fd71228 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:31.034769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:31.034769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:31.035043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:31.035625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] PATH_INFO: `/v3/users/dce62dc46d9b45c9b17198ffc40186a3` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:31.049940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dce62dc46d9b45c9b17198ffc40186a3)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:31.062744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:31.105621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:31.107104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:31.140929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 914/1803] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:31.163930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dce62dc46d9b45c9b17198ffc40186a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:31.185414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dce62dc46d9b45c9b17198ffc40186a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:31.218418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dce62dc46d9b45c9b17198ffc40186a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:31.221324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dce62dc46d9b45c9b17198ffc40186a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:31.266694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dce62dc46d9b45c9b17198ffc40186a3'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:31.275691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 890/1804] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:30 2022] DELETE /identity/v3/users/dce62dc46d9b45c9b17198ffc40186a3 => generated 0 bytes in 397 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:32.583900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:32.587413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:32.609138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79dfec-0dd1-4e7a-be58-5850614ffa8f service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:32.620918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:32.622569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:32.666681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:32.720915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:32.761008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:32.761981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:32.762549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af78b6-5164-4cb7-926e-60786c25be96 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:32.766788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:32.768486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] PATH_INFO: `/v3/projects/5e8ca740f70a4892b84a5b0abbb0ea44` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:32.771376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:32.772059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5e8ca740f70a4892b84a5b0abbb0ea44)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:32.772446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:32.772881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:32.773247 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] PATH_INFO: `/v3/projects/c60fbd06968e49b7a160539eeb9e0e09` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:32.777353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c60fbd06968e49b7a160539eeb9e0e09)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:32.777543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:32.783464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:32.915148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5e8ca740f70a4892b84a5b0abbb0ea44', 'request_id': 'req-20e09a80-f99f-4b11-891e-44cc78e81096'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.926803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5e8ca740f70a4892b84a5b0abbb0ea44', 'request_id': 'req-20e09a80-f99f-4b11-891e-44cc78e81096'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.934476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c60fbd06968e49b7a160539eeb9e0e09', 'request_id': 'req-d6c59654-f685-4d85-ba26-9074b16a96c7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.942422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5e8ca740f70a4892b84a5b0abbb0ea44', 'request_id': 'req-20e09a80-f99f-4b11-891e-44cc78e81096'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.944900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c60fbd06968e49b7a160539eeb9e0e09', 'request_id': 'req-d6c59654-f685-4d85-ba26-9074b16a96c7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.961782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c60fbd06968e49b7a160539eeb9e0e09', 'request_id': 'req-d6c59654-f685-4d85-ba26-9074b16a96c7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.971923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] The token cache is being invalidate because project 5e8ca740f70a4892b84a5b0abbb0ea44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:32.972762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.973735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-20e09a80-f99f-4b11-891e-44cc78e81096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.974809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] The token cache is being invalidate because project c60fbd06968e49b7a160539eeb9e0e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:32.975105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.975849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:32.982076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 891/1806] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:32 2022] DELETE /identity/v3/projects/c60fbd06968e49b7a160539eeb9e0e09 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:32.982076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:32.982076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:32.982998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 915/1806] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:32 2022] DELETE /identity/v3/projects/5e8ca740f70a4892b84a5b0abbb0ea44 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:32.986877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:32.987456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:32.988111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:32.990243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 916/1807] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:19:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:32.991774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:32.994931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.103810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937ee73a-d956-4c96-af35-b2332a7716b6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.109893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c59654-f685-4d85-ba26-9074b16a96c7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.115723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.117044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.117524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.117841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.118126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] PATH_INFO: `/v3/projects/8cdf00d9344943108f6407cac08d5f1a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.118685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.120542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8cdf00d9344943108f6407cac08d5f1a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.120686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.121676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] PATH_INFO: `/v3/users/58a853c4922a47fba91ac5bbb1b7e668` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.123228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.123553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=58a853c4922a47fba91ac5bbb1b7e668)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.126212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.184760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8cdf00d9344943108f6407cac08d5f1a', 'request_id': 'req-5bf50a4c-91e0-4c5c-80d3-d055aae47686'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.193597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8cdf00d9344943108f6407cac08d5f1a', 'request_id': 'req-5bf50a4c-91e0-4c5c-80d3-d055aae47686'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.199647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '58a853c4922a47fba91ac5bbb1b7e668'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.202651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:33.202771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8cdf00d9344943108f6407cac08d5f1a', 'request_id': 'req-5bf50a4c-91e0-4c5c-80d3-d055aae47686'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.211507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:33.218781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '58a853c4922a47fba91ac5bbb1b7e668'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.220227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '58a853c4922a47fba91ac5bbb1b7e668'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.221419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '58a853c4922a47fba91ac5bbb1b7e668'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.221540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] The token cache is being invalidate because project 8cdf00d9344943108f6407cac08d5f1a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:33.222139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.224659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.227627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 892/1808] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:32 2022] DELETE /identity/v3/projects/8cdf00d9344943108f6407cac08d5f1a => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:33.229362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:33.232717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.233563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4dbfa52-8df1-45a9-ac54-b9eb80e17b97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '58a853c4922a47fba91ac5bbb1b7e668'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.251213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 917/1809] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:32 2022] DELETE /identity/v3/users/58a853c4922a47fba91ac5bbb1b7e668 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:33.261541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff96d5d-d75d-4222-9b20-4c0d1ed7d83f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.262160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff96d5d-d75d-4222-9b20-4c0d1ed7d83f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.262822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff96d5d-d75d-4222-9b20-4c0d1ed7d83f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.291484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-cff96d5d-d75d-4222-9b20-4c0d1ed7d83f None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:33.292929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-cff96d5d-d75d-4222-9b20-4c0d1ed7d83f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.339339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf50a4c-91e0-4c5c-80d3-d055aae47686 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.395358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 918/1810] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:19:33 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:33.396782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169782-d650-443e-995c-d103bc4a92c7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.397075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169782-d650-443e-995c-d103bc4a92c7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.397345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f169782-d650-443e-995c-d103bc4a92c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.422780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.424724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.425246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.425627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] PATH_INFO: `/v3/users/5aea789eecdd40538f4fce67c110e943` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.429333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5aea789eecdd40538f4fce67c110e943)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.432869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.453923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7f169782-d650-443e-995c-d103bc4a92c7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:33.455504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7f169782-d650-443e-995c-d103bc4a92c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.471334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 919/1811] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:33.480726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.481006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.481275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.507251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:33.510535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.524186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 920/1812] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:33.531998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:33.541101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5aea789eecdd40538f4fce67c110e943'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.550779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5aea789eecdd40538f4fce67c110e943'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.551632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16971c42-f1f3-4d26-919c-15ba5a8d7c38 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.559433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.560054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.560561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.560838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.561773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:33.562082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.563713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.579591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5aea789eecdd40538f4fce67c110e943'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.585566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5aea789eecdd40538f4fce67c110e943'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.587523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 921/1813] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:19:33.590207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:33.593712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.618068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d8bb59-e7d1-4d36-8af9-7d4f91c6e69f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.629155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5aea789eecdd40538f4fce67c110e943'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:33.637073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.637686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.638411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.638691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.641388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.648531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 893/1814] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:33 2022] DELETE /identity/v3/users/5aea789eecdd40538f4fce67c110e943 => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:33.649844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:33.652677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.656808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.658776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.672307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 922/1815] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:33 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:33.704345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:33.792061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19474267-4efa-409d-9243-8a0ddd43ab29 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.809638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.810885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.811688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.813277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.815995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.818570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.820693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:33.823578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:33.853236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8e7536-6fd7-4ab2-aa9f-3f6c1a4eaa68 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:33.871591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:33.873531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:33.874742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:33.876697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] PATH_INFO: `/v3/projects/eace01176d9143c0a754ea9ebed6d34a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:33.880754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eace01176d9143c0a754ea9ebed6d34a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:33.908275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:33.924355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 894/1816] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:33 2022] POST /identity/v3/projects => generated 379 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:33.929433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.038592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eace01176d9143c0a754ea9ebed6d34a', 'request_id': 'req-f0a71403-2a14-4350-9d01-559b46d148a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.049767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eace01176d9143c0a754ea9ebed6d34a', 'request_id': 'req-f0a71403-2a14-4350-9d01-559b46d148a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.061135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eace01176d9143c0a754ea9ebed6d34a', 'request_id': 'req-f0a71403-2a14-4350-9d01-559b46d148a8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.076022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] The token cache is being invalidate because project eace01176d9143c0a754ea9ebed6d34a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:34.076631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.077624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f0a71403-2a14-4350-9d01-559b46d148a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.079581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 923/1817] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:33 2022] DELETE /identity/v3/projects/eace01176d9143c0a754ea9ebed6d34a => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:34.084426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.085261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.086587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.108269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:34.121200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d04b891-b161-4e2a-8ee6-4cd5445f816a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.138124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:34.145531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.292037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 924/1818] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:34.295432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.297125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.303065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.305956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.306926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.307483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.308747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.311272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6304dd2d-0ead-4c17-aac0-d80f223f157f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.344420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9a75e6-eeae-4436-b508-5e5454302f27 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.350394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.351876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.352338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.352756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] PATH_INFO: `/v3/projects/3eb3ad7255064fbea7968e327310fc0c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.354837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3eb3ad7255064fbea7968e327310fc0c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.357837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.361474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 895/1819] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:33 2022] POST /identity/v3/users => generated 290 bytes in 429 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:34.361474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.361474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.361474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.408259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:34.410174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.424427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 896/1820] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:34.424427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.426445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.445131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c689bb45-4486-4265-aedc-6f7318739a15 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.446113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3eb3ad7255064fbea7968e327310fc0c', 'request_id': 'req-a7256c71-6370-43d2-b33e-5f61bafaee21'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.455694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3eb3ad7255064fbea7968e327310fc0c', 'request_id': 'req-a7256c71-6370-43d2-b33e-5f61bafaee21'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.465633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3eb3ad7255064fbea7968e327310fc0c', 'request_id': 'req-a7256c71-6370-43d2-b33e-5f61bafaee21'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.500747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] The token cache is being invalidate because project 3eb3ad7255064fbea7968e327310fc0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:34.501232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.501811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:34.504003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 925/1821] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:34 2022] DELETE /identity/v3/projects/3eb3ad7255064fbea7968e327310fc0c => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:34.505057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.508163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.517673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:34.558244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.559062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.559347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.559625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.560579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.566175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2976be-2593-483a-813e-40d6dd0336d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.584252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 897/1822] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:34.589577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.589860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.590133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.627205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:34.628583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.639076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 898/1823] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:34.640673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.643417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.649366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7256c71-6370-43d2-b33e-5f61bafaee21 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.656551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.657298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.658103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.658422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.659288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:34.659661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.660947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd88140-f13a-4341-baa9-c661d25a8952 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.662195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd9f815a-c061-4a1e-97e2-74e811c8adf4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.667402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.668197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.668634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.669059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.669910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.671872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.683559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 926/1824] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:34.685280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.686832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.691358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.701555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 899/1825] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:34.703293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.718279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b10ab3-f70a-493a-89cf-ae9393abad9d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.721848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:34.723319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.724562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.725229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.725751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.726028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.726695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.728621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.732656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 927/1826] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:34.735184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.751338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab954a5-6830-4f6a-8dd2-6b2fff63613f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.759289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.762064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.762883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.763376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] PATH_INFO: `/v3/projects/ae3aa33f13bb47cdad9a30b578ebea7e/users/70a98fdcdbc94e6aa11ae03e0b3233d9/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.770275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 900/1827] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:34.771466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.780773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:34.785794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ae3aa33f13bb47cdad9a30b578ebea7e, user_id=70a98fdcdbc94e6aa11ae03e0b3233d9, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.790181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.821143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 928/1828] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:34 2022] PUT /identity/v3/projects/ae3aa33f13bb47cdad9a30b578ebea7e/users/70a98fdcdbc94e6aa11ae03e0b3233d9/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:34.822321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.855291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b348402-b148-46f4-aa1a-5eab86e9fc11 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.896871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6116aade-4671-4b12-b7b6-74f9d33a0b75 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.903250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.903880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.904158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.904432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.905365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.907342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe8fb8d-b178-4408-9c25-54c09b03dd1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.938399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 929/1829] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/users => generated 289 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:34.939543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.939839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.940106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.943102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.943735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.944159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.944442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.945101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:34.945439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.948022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.966632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 901/1830] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:34.967611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:34.978784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14fde-7f7f-425b-b329-e9c8966aaf8e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:34.981694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] MFA Rules not processed for user `70a98fdcdbc94e6aa11ae03e0b3233d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:34.983773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:34.983924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:34.984544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:34.984829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:34.985105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:34.985754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:34.987636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:34.999827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 902/1831] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.000685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.013751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9744e02c-bd12-4cec-818f-3e006e07ba65 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.019951 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.020654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.021005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.021349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.022549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.024972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.074521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 903/1832] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/projects => generated 377 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:35.074521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.084565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 930/1833] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:34 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:35.086017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.089301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:35.105245 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a74fee-0c41-414e-8b30-5d331de35286 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.184215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.185268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.186542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.186837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.187509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.190989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.197850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bee139b-931a-42fb-ac08-c21654ea18cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.203442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.204005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.204292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.204568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.205281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.207282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.222478 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 904/1834] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.223770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.232629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:35.235932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a78df3-e0a1-4f67-8be6-f8eee75de1de admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.242322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.243163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.243602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.244559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.245968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.248774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.258202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 931/1835] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/users => generated 289 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:35.259179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.281825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc52b4b-6d1e-4043-a80b-88dcbd83d3e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.291062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 905/1836] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/projects => generated 377 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:35.292547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.301929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.302655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.302952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.303213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] PATH_INFO: `/v3/projects/15aa5430422e4d0f9181884b58742b15/users/5f5875db99024e58a7c8b5d3009aefb0/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.351214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15aa5430422e4d0f9181884b58742b15, user_id=5f5875db99024e58a7c8b5d3009aefb0, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.355575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.375685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 932/1837] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:35 2022] PUT /identity/v3/projects/15aa5430422e4d0f9181884b58742b15/users/5f5875db99024e58a7c8b5d3009aefb0/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:35.376525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.397424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f868ed-4c9f-41bf-83f4-cca83c2ff8ca admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.464348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f525c9-a2ea-40a3-9507-12abc49bce39 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.470926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.471811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.472957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.473307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.474041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.476042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d037073-7b7c-43bd-90ca-fa7574fb1402 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.493818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.495436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.495932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.496212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.497145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.499620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.502719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 933/1838] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:35.504053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.504397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.504676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.516408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 906/1839] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.517585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.529924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bad43c0-9e6e-4ee3-94e5-dccf968821cf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.536597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.537332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.537617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.537891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.538586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.540356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.540973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] MFA Rules not processed for user `5f5875db99024e58a7c8b5d3009aefb0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:35.543070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:35.556005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 907/1840] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.557109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.570465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b069379-4c9c-4dd1-b2e8-6290e247ee63 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.587111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.588791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.589801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.590401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.591288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.606050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.609448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 908/1841] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.612533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.616675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:35.626031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e8be40-7dcb-418c-852b-436873eba5be admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.627135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 934/1842] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:35.631182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.644250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.644967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.645090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5038a64e-3978-42c8-810d-afa2296e4224 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.645355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.645690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.646448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.648357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.649927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.651451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.652353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.652733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] PATH_INFO: `/v3/projects/45d28973422049dcaa44e7a5a32790fd/users/f72b630d0caa440183af306043916e6a/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.664668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 909/1843] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:35.664668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.667565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:35.672692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] RBAC: Authorizing `identity:create_grant(project_id=45d28973422049dcaa44e7a5a32790fd, user_id=f72b630d0caa440183af306043916e6a, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.676577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.692215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 935/1844] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:35 2022] PUT /identity/v3/projects/45d28973422049dcaa44e7a5a32790fd/users/f72b630d0caa440183af306043916e6a/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:35.694630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:35.753273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:35.759275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eff26bc9-45fb-4bd8-a355-055ba98f56a6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.821917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.823112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.823515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.823862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.825756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508951f0-7974-4041-97fa-945aa98ff705 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:35.832189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:35.832968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.833318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.833974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] PATH_INFO: `/v3/projects/9f40018b369e4fb9a4b2dc691a71e207/users/23a173d485914ae5929117653c1839d9/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.855987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f40018b369e4fb9a4b2dc691a71e207, user_id=23a173d485914ae5929117653c1839d9, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.859099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-644dbde1-5d26-40a9-a98a-a4a24cec5cd8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.888506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 936/1845] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:35 2022] PUT /identity/v3/projects/9f40018b369e4fb9a4b2dc691a71e207/users/23a173d485914ae5929117653c1839d9/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:35.888506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.888506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.888506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.893023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:35.894475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89edc139-077e-42e2-b1bf-123d4f7451ac service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:35.906443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 910/1846] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:35 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 243 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:35.907842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:35.907963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:35.908319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:35.941873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] MFA Rules not processed for user `23a173d485914ae5929117653c1839d9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:35.943944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:35.956401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] MFA Rules not processed for user `f72b630d0caa440183af306043916e6a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:35.958339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:36.018682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:36.066574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 911/1847] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:36.069280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.119768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 937/1848] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:35 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:36.123522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.177827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d5c635-d117-43eb-9ccf-fcd50690f0a1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.184528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.185607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.186065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.186568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.187572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.190499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.233663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6827c1e7-3d21-436e-9616-8f36903e01bc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.241969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 912/1849] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:36 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:36.245250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.245965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.246284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.246553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.247409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.248807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.249273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.309249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 938/1850] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:36 2022] POST /identity/v3/projects => generated 381 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:36.313243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.374201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:36.377867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9daba9bb-d050-460b-95c7-db4b41ee0fb8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.417536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e540f980-4a06-430a-8b8b-d1b0920258ab admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.430578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.433724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.435138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.435420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.439186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.443522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.473662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 939/1851] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:36 2022] POST /identity/v3/users => generated 297 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:36.477743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.520918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6f8a39-63e3-423a-b1ef-91c8fd831c32 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.540172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.542892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.543174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.543518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.545037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.550444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.551206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.551550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.551814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.552460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.553198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.557134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.582761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 940/1852] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:36.589110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.606120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 913/1853] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:36 2022] POST /identity/v3/users => generated 295 bytes in 357 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:36.608281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.612476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f8e78e9-25f8-4691-895b-e0eb2d7a9328 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.636324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.638956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9947e17-7baf-4863-b5ff-0aabf24d6cb6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.639214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.639514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.639790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] PATH_INFO: `/v3/projects/c562447983ec48a1b705329533a4d36f/users/98512e62c7fd4941a9946ee76e2a592a/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.660581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.661136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.661382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.661660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.662338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.663950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.693379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c562447983ec48a1b705329533a4d36f, user_id=98512e62c7fd4941a9946ee76e2a592a, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.702357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 914/1854] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:36.702571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.704122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.714543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:36.727473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 941/1855] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:36 2022] PUT /identity/v3/projects/c562447983ec48a1b705329533a4d36f/users/98512e62c7fd4941a9946ee76e2a592a/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:36.732056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f20096b-9b22-4c25-867c-c54e220b2347 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.733303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.836816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd14fe4a-fc44-45ac-8725-25dd3f7565c9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.842957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.843634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.843916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.844190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.844849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.845028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.845606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.845887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.846164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] PATH_INFO: `/v3/projects/8b90bf9c488d47dabddd06c145141ed8/users/f47e5fa2e150412b9324ab53074b7619/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.847132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.867117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 942/1856] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:36.871324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.879182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b90bf9c488d47dabddd06c145141ed8, user_id=f47e5fa2e150412b9324ab53074b7619, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.882982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.884059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8ca541-9bcf-4a70-9171-05821a545d62 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:36.893064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:36.893801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.894083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.894387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] PATH_INFO: `/v3/projects/c562447983ec48a1b705329533a4d36f/users/98512e62c7fd4941a9946ee76e2a592a/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.897484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c562447983ec48a1b705329533a4d36f, user_id=98512e62c7fd4941a9946ee76e2a592a, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:36.902237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b09826-2337-443b-a38c-507ca58ac5a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:36.906753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 915/1857] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:36 2022] PUT /identity/v3/projects/8b90bf9c488d47dabddd06c145141ed8/users/f47e5fa2e150412b9324ab53074b7619/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:36.912513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:36.926214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 943/1858] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:36 2022] PUT /identity/v3/projects/c562447983ec48a1b705329533a4d36f/users/98512e62c7fd4941a9946ee76e2a592a/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:36.927829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:36.928160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:36.928397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:36.980945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] MFA Rules not processed for user `98512e62c7fd4941a9946ee76e2a592a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:36.984117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:37.016642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab079b3-7889-417b-aa33-2fa7c387e163 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.047048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:37.071947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 944/1859] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:37.083608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:37.086218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.086996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.087277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.087756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.089560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.091358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:37.105521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 916/1860] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:36 2022] GET /identity/v3/roles => generated 1420 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:37.107317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:37.121639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2494324d-dbd7-4c0b-8011-55b9bc63d4f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.126800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.127483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.129472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.129754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] PATH_INFO: `/v3/projects/8b90bf9c488d47dabddd06c145141ed8/users/f47e5fa2e150412b9324ab53074b7619/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.132065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b90bf9c488d47dabddd06c145141ed8, user_id=f47e5fa2e150412b9324ab53074b7619, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.135315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d666ba0f-ae6f-4ed4-97af-30ebd1368db9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:37.173852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 917/1861] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:37 2022] PUT /identity/v3/projects/8b90bf9c488d47dabddd06c145141ed8/users/f47e5fa2e150412b9324ab53074b7619/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:37.174505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fb8e491-6cd7-441d-9d7f-49973053a27b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.192243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.192831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.193359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.236889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] MFA Rules not processed for user `f47e5fa2e150412b9324ab53074b7619`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:37.238570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:37.245846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:37.249654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.250996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.251280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.251558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.328539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 918/1862] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:37 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:37.340147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.341980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:37.342107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:37.343270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:37.352427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 945/1863] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:37 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 269 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:37.357586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9938cb4a-755b-4fe0-bbe8-6aff29f44e53 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.361226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.362294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.362294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.362460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.373184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.375067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:37.388505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 919/1864] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:37 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:37.446707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:37.459518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413aef74-81a4-4046-aec9-f9e63f46a776 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.464259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.465025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.465307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.465820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.540716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:37.547644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.549180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54d0ba64-d8d1-4fc1-98e9-394e47ed24bc service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:37.563751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 946/1865] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:37.856117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:37.950172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47de6e-ec36-4a2c-9a22-64f48f18894e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:37.955054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:37.955764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:37.956116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:37.956480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] PATH_INFO: `/v3/users/f72b630d0caa440183af306043916e6a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:37.957841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f72b630d0caa440183af306043916e6a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:37.959813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:38.030658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f72b630d0caa440183af306043916e6a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.040344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f72b630d0caa440183af306043916e6a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.052081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f72b630d0caa440183af306043916e6a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.052731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f72b630d0caa440183af306043916e6a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.070232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f72b630d0caa440183af306043916e6a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.081081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 920/1866] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:37 2022] DELETE /identity/v3/users/f72b630d0caa440183af306043916e6a => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:38.085564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:38.086029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:38.086127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:38.107555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:38.108776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:38.191278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 947/1867] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:38.203769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:38.205278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:38.216807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:38.266067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38028424-dd4b-4655-9a71-67ad719c4634 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:38.270077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:38.270719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:38.270999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:38.271269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:38.280456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:38.281918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:38.291452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 921/1868] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:38 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:38.557120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:38.558646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:38.570476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12df5a38-dee2-4343-9bba-396ed81b3e48 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:38.576011 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:38.576726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:38.576931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:38.577415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] PATH_INFO: `/v3/projects/45d28973422049dcaa44e7a5a32790fd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:38.578456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] RBAC: Authorizing `identity:delete_project(project_id=45d28973422049dcaa44e7a5a32790fd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:38.580361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:38.629592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '45d28973422049dcaa44e7a5a32790fd', 'request_id': 'req-491953e1-fd34-48cb-9507-369efec01893'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.637528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '45d28973422049dcaa44e7a5a32790fd', 'request_id': 'req-491953e1-fd34-48cb-9507-369efec01893'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.648223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '45d28973422049dcaa44e7a5a32790fd', 'request_id': 'req-491953e1-fd34-48cb-9507-369efec01893'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.665786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] The token cache is being invalidate because project 45d28973422049dcaa44e7a5a32790fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:38.666330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.667501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.670198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 948/1869] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:38 2022] DELETE /identity/v3/projects/45d28973422049dcaa44e7a5a32790fd => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:38.676271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:38.693259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2eeace-62ea-44c7-b77e-8eb0e6808368 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:38.779398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:38.780044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:38.780334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:38.780610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] PATH_INFO: `/v3/users/98512e62c7fd4941a9946ee76e2a592a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:38.781761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98512e62c7fd4941a9946ee76e2a592a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:38.783715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:38.830288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98512e62c7fd4941a9946ee76e2a592a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.837234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98512e62c7fd4941a9946ee76e2a592a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.845109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98512e62c7fd4941a9946ee76e2a592a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.845780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98512e62c7fd4941a9946ee76e2a592a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.857963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-844c8419-55b9-48ce-a593-6c79af6e8eb5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98512e62c7fd4941a9946ee76e2a592a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:38.864069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 922/1870] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:38 2022] DELETE /identity/v3/users/98512e62c7fd4941a9946ee76e2a592a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:39.752888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:39.754389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:39.768770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491953e1-fd34-48cb-9507-369efec01893 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:39.830160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:39.830834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:39.831114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:39.831446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] PATH_INFO: `/v3/projects/c562447983ec48a1b705329533a4d36f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:39.832583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c562447983ec48a1b705329533a4d36f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:39.834504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:39.860243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:39.897152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c562447983ec48a1b705329533a4d36f', 'request_id': 'req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:39.904248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c562447983ec48a1b705329533a4d36f', 'request_id': 'req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:39.919075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c562447983ec48a1b705329533a4d36f', 'request_id': 'req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:39.933661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] The token cache is being invalidate because project c562447983ec48a1b705329533a4d36f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:39.933998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:39.934725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-ea2cd345-91a6-4210-b06a-b8d7f9c531d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:39.936489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 949/1871] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:39 2022] DELETE /identity/v3/projects/c562447983ec48a1b705329533a4d36f => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:39.941831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:39.942646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:39.943129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:39.967930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:39.969156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:39.998795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:40.042744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 923/1872] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:40.050441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.050734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.050993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.072244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:40.073737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:40.081734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 950/1873] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:40.086023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.111546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf95162-c97c-4152-848f-9eeb302b5c41 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.117578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.120473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.120930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.121390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.122067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:40.122432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.124085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.148671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 924/1874] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:40.153050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.167785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1caa59e-c313-4443-b67d-1e8d165c1696 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.173748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.174427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.174764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.175040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.175820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.179109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.211986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 951/1875] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:40.216981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.229010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edccc07a-acbb-4a5d-8e1e-acf08906e3d7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.307332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.308223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.308501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.308777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.309423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.311451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.334006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 925/1876] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/users => generated 292 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:40.338224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.348672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65a56f2-59d2-4c53-a81b-83a07cbe391a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.357943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.358665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.358939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.359204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.359841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.361411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.374837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 952/1877] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:40.379290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.393597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef3a69cb-04da-4e36-afc4-c7498592496a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.398628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.399265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.399544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.399817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.401349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.403405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3eb5b-3fb0-43fb-a6c3-4776287c60a9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.422059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 926/1878] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:40.423072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.442632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a8724a-76b9-4aba-ad6c-ee633ba8d5d7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.451521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.452933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.453433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.453923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] PATH_INFO: `/v3/projects/30c4109e0b744968a03ff9d560482b01/users/11f49b6a8c6e4e61a7c7848ef9e08a42/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.483906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30c4109e0b744968a03ff9d560482b01, user_id=11f49b6a8c6e4e61a7c7848ef9e08a42, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.488916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.506770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 953/1879] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:40 2022] PUT /identity/v3/projects/30c4109e0b744968a03ff9d560482b01/users/11f49b6a8c6e4e61a7c7848ef9e08a42/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:40.518122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.518436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.518798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.552735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] MFA Rules not processed for user `11f49b6a8c6e4e61a7c7848ef9e08a42`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:40.554146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:40.640686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 927/1880] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:40.645488 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.727053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55a00aa1-a393-4882-a5ae-da82d6e1275f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.735206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.736819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.737100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.737373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.738193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.740269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.777360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 954/1881] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/projects => generated 383 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:40.778106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.791070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc13c627-bfec-4213-b1f8-9430e5b7af3e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.829294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:40.872885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.873645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.873998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.874368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.875475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.877075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.907399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 928/1882] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:40 2022] POST /identity/v3/users => generated 298 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:40.911251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.924844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a016d2f-c53d-4220-aa25-b94d6b5dc97c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.931618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.932804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.933358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.933735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.934893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.936731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:40.950680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 955/1883] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:40 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:40.952667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:40.964943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e391d1-7e24-451b-b4a6-8e1d0b18cc13 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:40.970988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:40.971683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:40.971958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:40.972233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] PATH_INFO: `/v3/projects/60eb52d0529040cf988f2495b5a152c8/users/bbccc9abae5743c7883330de1f4184e9/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:40.989409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60eb52d0529040cf988f2495b5a152c8, user_id=bbccc9abae5743c7883330de1f4184e9, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:40.992115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.007538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 929/1884] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:40 2022] PUT /identity/v3/projects/60eb52d0529040cf988f2495b5a152c8/users/bbccc9abae5743c7883330de1f4184e9/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:41.012332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:41.028509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8323ba0-2383-43a8-9175-f807ab8db927 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:41.047657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:41.106720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:41.107352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.107636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.107913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.108562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:41.110167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6951a01e-561f-48c5-be67-cce837940c06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.121400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 956/1885] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:41 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:41.125552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:41.136795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32936dd0-ab0e-4d91-a01f-2fd0a2e24df6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:41.141475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:41.142530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.142898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.143237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] PATH_INFO: `/v3/projects/60eb52d0529040cf988f2495b5a152c8/users/bbccc9abae5743c7883330de1f4184e9/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.145239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=60eb52d0529040cf988f2495b5a152c8, user_id=bbccc9abae5743c7883330de1f4184e9, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:41.148181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.160010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 930/1886] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:41 2022] PUT /identity/v3/projects/60eb52d0529040cf988f2495b5a152c8/users/bbccc9abae5743c7883330de1f4184e9/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:41.170567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.170860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.171139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.200564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] MFA Rules not processed for user `bbccc9abae5743c7883330de1f4184e9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:41.201822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:41.271346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 957/1887] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:41 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:41.287241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:41.288947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:41.346409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46fca04-18ae-4372-8727-938db0c032d3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:41.350013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:41.350708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.350985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.351251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.360179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:41.361599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.371372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 931/1888] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:41 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:41.461205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:41.470864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79355aeb-9f0b-4647-996c-3ef6894b456e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:41.474469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:41.475065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.475343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.475615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.532869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:41.534202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a038938f-eb0d-44b3-8633-b7aeb5d9cccd service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.543557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 958/1889] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:41 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:41.816728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:41.880658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a3af74-f217-4823-8027-eba28b62a8df service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:41.885722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:41.886376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:41.886682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:41.886941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] PATH_INFO: `/v3/users/11f49b6a8c6e4e61a7c7848ef9e08a42` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:41.888096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=11f49b6a8c6e4e61a7c7848ef9e08a42)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:41.890021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:41.913193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:41.944072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11f49b6a8c6e4e61a7c7848ef9e08a42'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:41.956039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11f49b6a8c6e4e61a7c7848ef9e08a42'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:41.968932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11f49b6a8c6e4e61a7c7848ef9e08a42'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:41.969792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11f49b6a8c6e4e61a7c7848ef9e08a42'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:41.990052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11f49b6a8c6e4e61a7c7848ef9e08a42'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:41.999246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 932/1890] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:41 2022] DELETE /identity/v3/users/11f49b6a8c6e4e61a7c7848ef9e08a42 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:42.003684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488ccfa-84be-4e03-b063-1e6def6beab9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.004158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488ccfa-84be-4e03-b063-1e6def6beab9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.004251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4488ccfa-84be-4e03-b063-1e6def6beab9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.028181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4488ccfa-84be-4e03-b063-1e6def6beab9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:42.029411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4488ccfa-84be-4e03-b063-1e6def6beab9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:42.068842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:42.109945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 959/1891] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:42.121259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:42.122835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:42.182584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54309-4f08-4c9a-b079-cc93a3c38592 None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.182584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54309-4f08-4c9a-b079-cc93a3c38592 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.182879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b54309-4f08-4c9a-b079-cc93a3c38592 None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.187030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 960/1892] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:19:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:19:42.200146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.200530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.200888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.207767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc16b45-f837-47a1-92d9-f068b4ea76aa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:42.212269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:42.213931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.214421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.214796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.229594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:42.231251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:42.233375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:42.234822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:42.246843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 933/1893] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:42 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:42.324257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 961/1894] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:19:42 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:42.338775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:42.340606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:42.361559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b663a1-133d-421b-b547-41323eba64a0 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:42.367409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:42.368145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.368428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.368702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.379384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:42.380676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:42.394560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 934/1895] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:42 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:42.634311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:42.636463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:42.651662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d65fd8a-ce23-4dc7-b510-66989aab8cbb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:42.656373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:42.657009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.657287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.657555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] PATH_INFO: `/v3/projects/30c4109e0b744968a03ff9d560482b01` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.658873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=30c4109e0b744968a03ff9d560482b01)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:42.660757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:42.710995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30c4109e0b744968a03ff9d560482b01', 'request_id': 'req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.718582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30c4109e0b744968a03ff9d560482b01', 'request_id': 'req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.727794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30c4109e0b744968a03ff9d560482b01', 'request_id': 'req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.742323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] The token cache is being invalidate because project 30c4109e0b744968a03ff9d560482b01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:42.742718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.743450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.746413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 962/1896] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:42 2022] DELETE /identity/v3/projects/30c4109e0b744968a03ff9d560482b01 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:42.750573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:42.764121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19bad06c-4f44-44e2-8834-f5895f5fde9a service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:42.842209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:42.842884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:42.843318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:42.843594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] PATH_INFO: `/v3/users/bbccc9abae5743c7883330de1f4184e9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:42.844779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbccc9abae5743c7883330de1f4184e9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:42.846769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:42.895656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:42.915318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbccc9abae5743c7883330de1f4184e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.925918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbccc9abae5743c7883330de1f4184e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.937391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbccc9abae5743c7883330de1f4184e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.937391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbccc9abae5743c7883330de1f4184e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.955132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3c9e4c9-65df-46dc-a045-89d1809a46c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbccc9abae5743c7883330de1f4184e9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:42.964008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 935/1897] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:42 2022] DELETE /identity/v3/users/bbccc9abae5743c7883330de1f4184e9 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:43.435401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:43.436948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:43.458019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5b3acb-5a30-4ee2-8b2b-915353f996e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:43.635308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:43.654744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:43.655526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:43.655976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:43.656396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] PATH_INFO: `/v3/projects/60eb52d0529040cf988f2495b5a152c8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:43.658768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] RBAC: Authorizing `identity:delete_project(project_id=60eb52d0529040cf988f2495b5a152c8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:43.660744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:43.734932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '60eb52d0529040cf988f2495b5a152c8', 'request_id': 'req-3d002863-2cb5-4ffc-8cb6-f163cb92d166'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:43.746574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '60eb52d0529040cf988f2495b5a152c8', 'request_id': 'req-3d002863-2cb5-4ffc-8cb6-f163cb92d166'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:43.757426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '60eb52d0529040cf988f2495b5a152c8', 'request_id': 'req-3d002863-2cb5-4ffc-8cb6-f163cb92d166'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:43.777335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] The token cache is being invalidate because project 60eb52d0529040cf988f2495b5a152c8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:43.777639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:43.778891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:43.782305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 963/1898] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:43 2022] DELETE /identity/v3/projects/60eb52d0529040cf988f2495b5a152c8 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:43.785363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:43.785658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:43.785933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:43.809248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:43.810851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:43.842442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:43.844131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:43.908526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 936/1899] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:43.909487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:43.912435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:43.925155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb884120-751a-4265-ae61-481c6517ef37 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:43.930527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:43.931411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:43.932311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:43.932530 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] PATH_INFO: `/v3/users/70a98fdcdbc94e6aa11ae03e0b3233d9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:43.933926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=70a98fdcdbc94e6aa11ae03e0b3233d9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:43.937501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:43.940690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d002863-2cb5-4ffc-8cb6-f163cb92d166 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:43.946386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:43.948434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:43.948740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:43.949016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] PATH_INFO: `/v3/users/bdce1a7433a344599533213dc17f4878` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:43.950334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bdce1a7433a344599533213dc17f4878)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:43.952284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:43.995295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '70a98fdcdbc94e6aa11ae03e0b3233d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.003733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '70a98fdcdbc94e6aa11ae03e0b3233d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.014713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '70a98fdcdbc94e6aa11ae03e0b3233d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.015913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '70a98fdcdbc94e6aa11ae03e0b3233d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.023186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bdce1a7433a344599533213dc17f4878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.037291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e677118-b402-4748-833a-15173b21af85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '70a98fdcdbc94e6aa11ae03e0b3233d9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.040591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bdce1a7433a344599533213dc17f4878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.042614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bdce1a7433a344599533213dc17f4878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.043453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bdce1a7433a344599533213dc17f4878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.049809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 937/1900] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:43 2022] DELETE /identity/v3/users/70a98fdcdbc94e6aa11ae03e0b3233d9 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:44.052370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb027d5-ef64-4e12-9512-4684d2457304 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.052682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb027d5-ef64-4e12-9512-4684d2457304 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.053317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb027d5-ef64-4e12-9512-4684d2457304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.059202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c283d49a-a077-4ce9-af90-0827f11a3133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bdce1a7433a344599533213dc17f4878'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:44.075922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 964/1901] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:43 2022] DELETE /identity/v3/users/bdce1a7433a344599533213dc17f4878 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:44.075922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.076183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.076255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.080288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-efb027d5-ef64-4e12-9512-4684d2457304 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:44.081765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-efb027d5-ef64-4e12-9512-4684d2457304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.084358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:44.100349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:44.101188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:44.105052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.172409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 938/1902] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:44.173993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.174459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.174888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.179456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 965/1903] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:44.186759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.204409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e00d2b6-6ef4-44e3-ade9-5f8c13a4ec84 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.208618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:44.209512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.210344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.210573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.210991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.211985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:44.212192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.212482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.215601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.225274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 939/1904] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:44.227048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.231544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.248337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 966/1905] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:44.251072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.253180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.296140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0152f5b-ee18-4175-bedd-b849eae7aeb2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.300263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.300983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.301414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.302019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.312042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.313738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.314115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcf4c0-ea1b-43bb-b26d-ddc2a82f68be admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.318176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.319283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.319633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.320508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.324739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 940/1906] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:44.325542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.331264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.332689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.343487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da27e473-84c8-486a-b065-a12b9f56ac7b service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.348813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.349832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.350114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.350427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.351068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.353178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.358924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 967/1907] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:44.386502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 941/1908] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/projects => generated 383 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:44.390703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.400904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7225a3-eb93-4bee-ac48-9bd8c2c1f3c0 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.461321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.462024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.462207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.462529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.463201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.465122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.489855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 968/1909] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:44.491163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.502054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e53d6c-7978-4657-9ecb-f95b3ed6361f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.507005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.507773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.508125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.508466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.509177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.511496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.525163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 942/1910] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:44.527163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.537276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf06928-8343-4071-ab64-be5263c43f1e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.542708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.543391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.543740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.544084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.544799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.546620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109e5d09-c150-41ea-8066-75fcf74e4889 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.561523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 969/1911] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:44.562806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.573873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e5fc8a-2b71-4756-99cd-540ceb5c6a3d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.579581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.580310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.580641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.585811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] PATH_INFO: `/v3/projects/54c4d8946263464fbde1c3c1f894f482/users/a11270ee6e7c4d8c8cb27727e26872ed/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.598923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] RBAC: Authorizing `identity:create_grant(project_id=54c4d8946263464fbde1c3c1f894f482, user_id=a11270ee6e7c4d8c8cb27727e26872ed, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.604804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.639419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 943/1912] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:44 2022] PUT /identity/v3/projects/54c4d8946263464fbde1c3c1f894f482/users/a11270ee6e7c4d8c8cb27727e26872ed/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:44.641469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.641756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.642306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.663930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:44.677520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] MFA Rules not processed for user `a11270ee6e7c4d8c8cb27727e26872ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:44.679661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.757183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 970/1913] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:44 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:44.771049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.772943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.842961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5621fbe-07a6-4ab5-bbef-2d1dfd3ca9fe admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.847049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.847722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.848001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.848357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.857970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.859449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.868453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 944/1914] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:44 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:44.888857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.958408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:44.960184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:44.964449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d4c26b-93c4-4c57-b857-bc969bb3169c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.975236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:44.980610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.981419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.981803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.983913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] PATH_INFO: `/v3/users/a11270ee6e7c4d8c8cb27727e26872ed` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.987000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] RBAC: Authorizing `identity:delete_user(user_id=a11270ee6e7c4d8c8cb27727e26872ed)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:44.990106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c8f9532-dbdf-4481-9e92-bb74af4bf18f service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:44.991041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:44.996484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:44.997121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:44.997419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:44.997687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] PATH_INFO: `/v3/projects/ae3aa33f13bb47cdad9a30b578ebea7e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:44.998890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ae3aa33f13bb47cdad9a30b578ebea7e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.000784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.046759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a11270ee6e7c4d8c8cb27727e26872ed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.052839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ae3aa33f13bb47cdad9a30b578ebea7e', 'request_id': 'req-335f4a0b-6516-4f27-a37e-f1489060a603'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.062339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ae3aa33f13bb47cdad9a30b578ebea7e', 'request_id': 'req-335f4a0b-6516-4f27-a37e-f1489060a603'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.064184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a11270ee6e7c4d8c8cb27727e26872ed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.065611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a11270ee6e7c4d8c8cb27727e26872ed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.066275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a11270ee6e7c4d8c8cb27727e26872ed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.069206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ae3aa33f13bb47cdad9a30b578ebea7e', 'request_id': 'req-335f4a0b-6516-4f27-a37e-f1489060a603'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.076394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a11270ee6e7c4d8c8cb27727e26872ed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.082194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] The token cache is being invalidate because project ae3aa33f13bb47cdad9a30b578ebea7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:45.082522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.083185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.084711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 945/1915] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:44 2022] DELETE /identity/v3/projects/ae3aa33f13bb47cdad9a30b578ebea7e => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.087792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.087792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.093357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 971/1916] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:44 2022] DELETE /identity/v3/users/a11270ee6e7c4d8c8cb27727e26872ed => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.096289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.096289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.118129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:45.156027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335f4a0b-6516-4f27-a37e-f1489060a603 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.160631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:45.161255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.161532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.161799 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] PATH_INFO: `/v3/projects/fd0f64a24a554a74a3b21e37aaf7e966` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.162932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd0f64a24a554a74a3b21e37aaf7e966)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.164806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.167229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-194db730-a4e6-4e94-ab60-f5dbf4ceb67e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.172114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:45.172820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.173470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.173748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] PATH_INFO: `/v3/users/23a173d485914ae5929117653c1839d9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.174960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23a173d485914ae5929117653c1839d9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.176892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.216342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd0f64a24a554a74a3b21e37aaf7e966', 'request_id': 'req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.224926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd0f64a24a554a74a3b21e37aaf7e966', 'request_id': 'req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.226284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23a173d485914ae5929117653c1839d9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.231446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd0f64a24a554a74a3b21e37aaf7e966', 'request_id': 'req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.238710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23a173d485914ae5929117653c1839d9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.239514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23a173d485914ae5929117653c1839d9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.240561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23a173d485914ae5929117653c1839d9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.241719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] The token cache is being invalidate because project fd0f64a24a554a74a3b21e37aaf7e966 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:45.242003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.242678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d4f3664c-67c7-49a0-8b7b-1af21f84c34c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.244797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 946/1917] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/projects/fd0f64a24a554a74a3b21e37aaf7e966 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.248755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23a173d485914ae5929117653c1839d9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.248906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.249063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.249328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.264326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 972/1918] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/users/23a173d485914ae5929117653c1839d9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.264964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.266345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.272404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:45.274730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.287932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:45.383017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 947/1919] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:45.385600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54e3a45-1d2e-4ab6-931d-3da7688e1f90 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.393129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:45.393794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.394065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.394179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.394389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] PATH_INFO: `/v3/users/d21ef229781247f1a930a26367519741` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.395732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d21ef229781247f1a930a26367519741)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.395838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.398092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.471159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:45.485156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd21ef229781247f1a930a26367519741'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.493840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdfd352-2cb3-44ad-bc93-9a00fafbf473 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.502397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:45.504262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.509070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.510149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.517845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd21ef229781247f1a930a26367519741'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.519092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd21ef229781247f1a930a26367519741'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.520250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd21ef229781247f1a930a26367519741'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.529695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.531338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.533172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd21ef229781247f1a930a26367519741'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.543976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 948/1920] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:45 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:45.552397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 973/1921] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/users/d21ef229781247f1a930a26367519741 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.684636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.688274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.711623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8818c4d-df07-4edd-bc23-d46b7cf43941 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.800972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:45.801812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:45.802197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:45.802602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] PATH_INFO: `/v3/projects/9f40018b369e4fb9a4b2dc691a71e207` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:45.804628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f40018b369e4fb9a4b2dc691a71e207)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:45.810898 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:45.883588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.886067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:45.910598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f40018b369e4fb9a4b2dc691a71e207', 'request_id': 'req-b907c31e-afbd-4b5c-acf0-2c55afbd4566'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.920246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2337aa95-06a8-4bbc-aebf-278e10de1915 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:45.921258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f40018b369e4fb9a4b2dc691a71e207', 'request_id': 'req-b907c31e-afbd-4b5c-acf0-2c55afbd4566'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.932839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f40018b369e4fb9a4b2dc691a71e207', 'request_id': 'req-b907c31e-afbd-4b5c-acf0-2c55afbd4566'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.938602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:45.948140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] The token cache is being invalidate because project 9f40018b369e4fb9a4b2dc691a71e207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:45.948648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.949684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:45.952162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 949/1922] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/projects/9f40018b369e4fb9a4b2dc691a71e207 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:45.967438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:45.970536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.030941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.031756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.032238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.032505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] PATH_INFO: `/v3/projects/54c4d8946263464fbde1c3c1f894f482` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.033656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54c4d8946263464fbde1c3c1f894f482)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.035591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.056344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b907c31e-afbd-4b5c-acf0-2c55afbd4566 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.063934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.064704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.065012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.065380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] PATH_INFO: `/v3/users/f47e5fa2e150412b9324ab53074b7619` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.066794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f47e5fa2e150412b9324ab53074b7619)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.069205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.098032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54c4d8946263464fbde1c3c1f894f482', 'request_id': 'req-d1152930-1f2f-499a-a367-15c1ada741a9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.101201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54c4d8946263464fbde1c3c1f894f482', 'request_id': 'req-d1152930-1f2f-499a-a367-15c1ada741a9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.113207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54c4d8946263464fbde1c3c1f894f482', 'request_id': 'req-d1152930-1f2f-499a-a367-15c1ada741a9'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.127956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] The token cache is being invalidate because project 54c4d8946263464fbde1c3c1f894f482 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:46.128181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.128960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d1152930-1f2f-499a-a367-15c1ada741a9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.131093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 974/1923] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/projects/54c4d8946263464fbde1c3c1f894f482 => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:46.136455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ae894-5f72-4591-b010-dffcddabbbda None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.137431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ae894-5f72-4591-b010-dffcddabbbda None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.137709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226ae894-5f72-4591-b010-dffcddabbbda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.139270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f47e5fa2e150412b9324ab53074b7619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.148868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f47e5fa2e150412b9324ab53074b7619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.159193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f47e5fa2e150412b9324ab53074b7619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.160547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f47e5fa2e150412b9324ab53074b7619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.162271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-226ae894-5f72-4591-b010-dffcddabbbda None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:46.163537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-226ae894-5f72-4591-b010-dffcddabbbda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.176196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f47e5fa2e150412b9324ab53074b7619'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.183080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 950/1924] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:45 2022] DELETE /identity/v3/users/f47e5fa2e150412b9324ab53074b7619 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:46.186100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.186100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.216687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:46.238321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 975/1925] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:46.246394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.246966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.247330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.272427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:46.274476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.278171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6752142c-c56b-47e1-9366-8a542c189824 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.285445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.286291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.286650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.286976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] PATH_INFO: `/v3/projects/3720f53142dd4af69c9df022291cb870` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.289288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3720f53142dd4af69c9df022291cb870)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.292054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.295842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 976/1926] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:46.296743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.305517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:46.307507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1457141-4447-4e98-b78b-af86bc8ddd69 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.313792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.314444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.314749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.314995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.315643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:46.316096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.317918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.335129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 977/1927] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:46.336272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.346761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8a51c-175c-484b-8dfc-51b0b73b1e6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.357150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3720f53142dd4af69c9df022291cb870', 'request_id': 'req-22b64268-9b31-4b78-b698-c82f8797da79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.370109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3720f53142dd4af69c9df022291cb870', 'request_id': 'req-22b64268-9b31-4b78-b698-c82f8797da79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.377377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3720f53142dd4af69c9df022291cb870', 'request_id': 'req-22b64268-9b31-4b78-b698-c82f8797da79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.388769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] The token cache is being invalidate because project 3720f53142dd4af69c9df022291cb870 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:46.389066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.389782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-22b64268-9b31-4b78-b698-c82f8797da79 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.391509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 951/1928] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:46 2022] DELETE /identity/v3/projects/3720f53142dd4af69c9df022291cb870 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:46.396366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76215a-1f4d-4869-95e3-5525b233b9c7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.396864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76215a-1f4d-4869-95e3-5525b233b9c7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.397142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff76215a-1f4d-4869-95e3-5525b233b9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.416622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.417208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.417630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.417909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.418699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.420771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.423531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ff76215a-1f4d-4869-95e3-5525b233b9c7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:46.424750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ff76215a-1f4d-4869-95e3-5525b233b9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.435118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 952/1929] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:46.443259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.443577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.443885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.451675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 978/1930] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:46.455993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.457711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.467597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:46.468755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.544677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 953/1931] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:46.544677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.545067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.564161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353f56cd-ebf9-46f9-a9af-1a2f63a7a1c1 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.577734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.579087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.580497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.580848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] PATH_INFO: `/v3/projects/8b90bf9c488d47dabddd06c145141ed8` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.584142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b90bf9c488d47dabddd06c145141ed8)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.586741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.593493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49490810-3dfa-46df-8919-fa3978e8c361 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.601900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.603628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.604201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.604522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.605202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.607197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.647923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 979/1932] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/users => generated 293 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:46.656791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.686281 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04671e27-1c18-43a8-b764-78494ca203b1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.703900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b90bf9c488d47dabddd06c145141ed8', 'request_id': 'req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.725556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b90bf9c488d47dabddd06c145141ed8', 'request_id': 'req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.741636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b90bf9c488d47dabddd06c145141ed8', 'request_id': 'req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.774627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] The token cache is being invalidate because project 8b90bf9c488d47dabddd06c145141ed8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:46.774912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.776752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:46.778649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 954/1933] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:46 2022] DELETE /identity/v3/projects/8b90bf9c488d47dabddd06c145141ed8 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:46.779522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.784427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.788185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:46.813237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.814137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.814767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.815301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.816951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:46.818457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.823141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86ea476-deac-4a4e-aff0-12def2bf1fad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.834304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:46.848212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 980/1934] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:46.848212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.848212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.848212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.871598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:46.873271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.884889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 981/1935] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:46.886417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.888760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:46.906361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1814e1c4-0dbd-40cc-b130-8669419e4c8f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.912706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.913285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.913571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.913841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.914512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.916823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5968e-92b8-4921-b5d5-d7267767793e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.944222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9937df47-f2b0-4ece-a2da-ec6cbd9902eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.950643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:46.951282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.951963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.952798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.953842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:46.955629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:46.966484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 982/1936] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/projects => generated 369 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:46.967887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:46.968176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:46.968449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:46.975263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 955/1937] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:46 2022] GET /identity/v3/roles => generated 1420 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:46.976358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:46.986963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29eb7b-fc09-49c4-bb1e-c151db80f545 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:46.999558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:47.001352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:47.059633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.060382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.060869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.061149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.061784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.063753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.086415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 983/1938] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:47.087731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.091328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 956/1939] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:46 2022] POST /identity/v3/users => generated 285 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:47.092082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.104551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7b356c9-0e77-439a-8133-69ba24039828 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.112021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.112187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda80b69-34eb-43e3-b8dc-68436cb7e01e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.112656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.113075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.113548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.114620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.116452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.118637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.120797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.121096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.121526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.122202 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:47.122569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.124906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.130981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 984/1940] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:47.131793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.145599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d97a51-8d13-4d52-8b8c-86f0881e11d5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.146027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 957/1941] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:47.146890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.155835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.156547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.156856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.157186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.157882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c082447-782a-4ffd-a4f5-3dbf58eeb970 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.158015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.159743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.163204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.163872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.164138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.164354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] PATH_INFO: `/v3/projects/de117a6ecbae4791a29c165fc99d9089/users/a0caac57a68a4262880ad77475e6b7ff/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.179521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 985/1942] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:47.180084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.184541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] RBAC: Authorizing `identity:create_grant(project_id=de117a6ecbae4791a29c165fc99d9089, user_id=a0caac57a68a4262880ad77475e6b7ff, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.187511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.196657 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfd7e39-2a8e-4dc1-91bc-e35ff1263fcc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.206238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.206640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.207060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.207335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.207844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 958/1943] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:47 2022] PUT /identity/v3/projects/de117a6ecbae4791a29c165fc99d9089/users/a0caac57a68a4262880ad77475e6b7ff/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:47.208938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.209522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.211793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f41125e8-7288-4ce0-8000-ff75eedb73ab admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.252851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 986/1944] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/projects => generated 387 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:47.254328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.254692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.254951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.289732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] MFA Rules not processed for user `a0caac57a68a4262880ad77475e6b7ff`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:47.291085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:47.307754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1bed14-5e92-46c1-9cc0-606c4b82b8bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.316870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:47.350734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:47.364336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 987/1945] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:47.365349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.373071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.374024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.374504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.374846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.375764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.378320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.389409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 959/1946] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:47.390091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.391762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:47.437958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350effb-4525-4a9c-a161-b4f1a1b6ab4e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.442618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.443238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.443640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.444387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.445430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.448025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.452440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaa3995-3e8a-4940-a468-07e764daa878 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.458452 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.459372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.459908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.460359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.472729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.474807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.478667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 988/1947] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/users => generated 294 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:47.478667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.488547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b661612-f532-4683-9509-c30b0d45ca92 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.492714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 960/1948] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:47.494666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.504268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.504986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.505263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.505552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] PATH_INFO: `/v3/projects/e1f16d1044c54aa99da6591e604d076e/users/06a5517bce6d40c6915544628a551910/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.511101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd817dfd-cdc0-4826-bfd7-2775fcf83e92 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.521737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.522847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.523205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.523553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.524843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.528451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.546270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1f16d1044c54aa99da6591e604d076e, user_id=06a5517bce6d40c6915544628a551910, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.548678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 961/1949] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:47.549410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9dee78-c497-4bcb-a52f-5b0bd093ac06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.549794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.566446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3247509d-d85e-48fc-a42e-32f3db20335d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.576069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 989/1950] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:47 2022] PUT /identity/v3/projects/e1f16d1044c54aa99da6591e604d076e/users/06a5517bce6d40c6915544628a551910/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:47.587009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.587295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.588102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.629550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] MFA Rules not processed for user `06a5517bce6d40c6915544628a551910`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:47.630962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:47.682860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.683837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.684358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.684850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.685860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.700927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.720711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 962/1951] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:47 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:47.726140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.732279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 990/1952] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:47.736463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.753605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4709d56f-3634-4149-9e9f-1974721ff266 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.759075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.760281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.760556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.760822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] PATH_INFO: `/v3/projects/2d4c50a599e44b2c8416385d81ed7cbb/users/835bbf05b5404926846f603f92349c5d/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.761035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a93670-0757-44e1-b2cf-1f6e8d0a8e52 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:47.768905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:47.769682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.769961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.770579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.771399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.774137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.796238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d4c50a599e44b2c8416385d81ed7cbb, user_id=835bbf05b5404926846f603f92349c5d, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:47.799158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02bbc53-9557-432c-b014-d5832c388db7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:47.819027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 991/1953] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/projects => generated 369 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:47.828308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 963/1954] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:47 2022] PUT /identity/v3/projects/2d4c50a599e44b2c8416385d81ed7cbb/users/835bbf05b5404926846f603f92349c5d/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:47.828489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:47.840417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:47.840780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:47.841184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:47.878615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:47.895097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] MFA Rules not processed for user `835bbf05b5404926846f603f92349c5d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:47.897112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:47.906220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:47.986493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2085d380-5622-4145-93d2-68f169f1fe65 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.009311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.010739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.011119 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.011417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.012067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.014028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.063883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 992/1955] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/users => generated 291 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:48.065233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 964/1956] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:47 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:48.069498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.071190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.083954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b13c852-1335-49c2-8331-3ac38ad7896d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.084854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74fdca2a-ffe0-4c7d-b932-0e8f4e22442f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.090329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.090961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.091294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.091580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.092276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.093441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.093927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.094518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.094932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.095732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.097208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.099665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.109567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 993/1957] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:48.112569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.125861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e671c9-cf35-42ff-bb63-d066109b5ebb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.133288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.133983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.134104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.134447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] PATH_INFO: `/v3/projects/de3af8e838d94f5ca76305f9ef3ce955/users/cf713100a4404af6abf04f241149441e/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.146212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 965/1958] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:48 2022] POST /identity/v3/projects => generated 385 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:48.148062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.156750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3af8e838d94f5ca76305f9ef3ce955, user_id=cf713100a4404af6abf04f241149441e, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.160475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.182673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 994/1959] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:48 2022] PUT /identity/v3/projects/de3af8e838d94f5ca76305f9ef3ce955/users/cf713100a4404af6abf04f241149441e/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:48.199448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.250087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abf848f-0348-4810-9fd8-f35e7f719815 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.307510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbdb648-1f41-4617-b764-7c11d598ee90 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.313564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.314178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.314497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.314771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.315502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.317165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.329027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 995/1960] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:48.333025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.344900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce634954-de13-4de1-a238-9c98429d6c47 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.350231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.351000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.351697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.351697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] PATH_INFO: `/v3/projects/de3af8e838d94f5ca76305f9ef3ce955/users/cf713100a4404af6abf04f241149441e/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.353751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] RBAC: Authorizing `identity:create_grant(project_id=de3af8e838d94f5ca76305f9ef3ce955, user_id=cf713100a4404af6abf04f241149441e, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.356574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b485953a-560a-47b8-bbdf-8c495c59a929 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.362680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.363398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.363740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.364069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.364762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.366832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.371270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 996/1961] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:48 2022] PUT /identity/v3/projects/de3af8e838d94f5ca76305f9ef3ce955/users/cf713100a4404af6abf04f241149441e/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:48.383975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.385067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.385452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.407514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 966/1962] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:48 2022] POST /identity/v3/users => generated 299 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:48.412879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.422848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] MFA Rules not processed for user `cf713100a4404af6abf04f241149441e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:48.424548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:48.425927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff2ab62-1043-4cc9-ba59-4fdf3cb02c34 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.441528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:48.485047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:48.509771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.509993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 997/1963] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:48.510178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.510484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.510778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.511479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.513210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.523994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.525307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 967/1964] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:48.529420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.540259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e81807f1-491b-4e27-a1ca-c15e2d056852 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.546654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.547403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.547733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.548113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] PATH_INFO: `/v3/projects/8ffcd14987ec4e97a037be822fd7c372/users/f3f6267abb774d209f2cf6ae05f615c6/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.566334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffcd14987ec4e97a037be822fd7c372, user_id=f3f6267abb774d209f2cf6ae05f615c6, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.569205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.584030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 968/1965] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:48 2022] PUT /identity/v3/projects/8ffcd14987ec4e97a037be822fd7c372/users/f3f6267abb774d209f2cf6ae05f615c6/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:48.588093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.605548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ffdf1d-7aa7-497f-9852-874c9cb5489d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.675453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2647d2-5e3c-4ded-98bb-18fbfb5fc820 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.680376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.680954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.682017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.682017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.682017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.682540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.682829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.682944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.683172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.684451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.695015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 969/1966] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:48.699618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.710150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacccfb1-9abb-46a5-8104-ee1cb37c5f04 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.719086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.720028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.720710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.720710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] PATH_INFO: `/v3/projects/8ffcd14987ec4e97a037be822fd7c372/users/f3f6267abb774d209f2cf6ae05f615c6/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.723055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ffcd14987ec4e97a037be822fd7c372, user_id=f3f6267abb774d209f2cf6ae05f615c6, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.726330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8af49be3-b3d6-4d97-895e-a8a752dd7d97 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.739547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 970/1967] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:48 2022] PUT /identity/v3/projects/8ffcd14987ec4e97a037be822fd7c372/users/f3f6267abb774d209f2cf6ae05f615c6/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:48.750509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.750826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.751051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.756732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.758157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.769766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 998/1968] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:48.783882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] MFA Rules not processed for user `f3f6267abb774d209f2cf6ae05f615c6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:48.785111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:48.865844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:48.882388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 971/1969] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:48 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:48.893212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:48.894833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:48.912357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1efa41df-c305-4fa5-bb8d-ec3462ebf5c0 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:48.970729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:48.971433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:48.971816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:48.972157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:48.988496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:48.989990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950b08e8-1cff-4226-a155-1d1beeeb5e3f service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:48.999899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:49.002361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 999/1970] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:48 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:50.062949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:50.131778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9492b0c5-efe7-4e9d-8339-2ee33ae99f31 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:50.136282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:50.137033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:50.137318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:50.137821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] PATH_INFO: `/v3/users/835bbf05b5404926846f603f92349c5d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:50.139099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] RBAC: Authorizing `identity:delete_user(user_id=835bbf05b5404926846f603f92349c5d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:50.141108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:50.191396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '835bbf05b5404926846f603f92349c5d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:50.199516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '835bbf05b5404926846f603f92349c5d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:50.208543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '835bbf05b5404926846f603f92349c5d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:50.209267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '835bbf05b5404926846f603f92349c5d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:50.223008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '835bbf05b5404926846f603f92349c5d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:50.230986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 972/1971] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:50 2022] DELETE /identity/v3/users/835bbf05b5404926846f603f92349c5d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:50.235612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:50.236083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:50.236265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:50.255904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:50.257212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:50.330369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1000/1972] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:50.341409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:50.343415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:50.435392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275f8846-303a-4b67-a046-8053de5dcf37 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:50.439179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:50.439784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:50.440064 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:50.440333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:50.460666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:50.463174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:50.477526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 973/1973] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:50.899522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:50.902368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:50.921500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce19736-70d6-412e-a101-85a34f33de89 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:50.927748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:50.928765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:50.929145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:50.929586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] PATH_INFO: `/v3/users/06a5517bce6d40c6915544628a551910` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:50.931687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] RBAC: Authorizing `identity:delete_user(user_id=06a5517bce6d40c6915544628a551910)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:50.934328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.008105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.009895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.021913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06a5517bce6d40c6915544628a551910'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.032145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d22c21-ad17-4fae-bbd0-66cb6e57b6e5 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.049207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:51.049988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.050412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.050854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] PATH_INFO: `/v3/projects/2d4c50a599e44b2c8416385d81ed7cbb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.052300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d4c50a599e44b2c8416385d81ed7cbb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:51.053811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06a5517bce6d40c6915544628a551910'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.055277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.065123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06a5517bce6d40c6915544628a551910'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.066568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06a5517bce6d40c6915544628a551910'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.070980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:51.079147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-91252218-373e-4fc1-9829-da7062c3f441 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06a5517bce6d40c6915544628a551910'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.094646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1001/1974] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:50 2022] DELETE /identity/v3/users/06a5517bce6d40c6915544628a551910 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:51.099795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.100833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.103448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.141676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d4c50a599e44b2c8416385d81ed7cbb', 'request_id': 'req-05972e66-4e7c-44e8-a6c0-d115c0a734f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.145054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:51.146839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.153510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d4c50a599e44b2c8416385d81ed7cbb', 'request_id': 'req-05972e66-4e7c-44e8-a6c0-d115c0a734f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.171551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d4c50a599e44b2c8416385d81ed7cbb', 'request_id': 'req-05972e66-4e7c-44e8-a6c0-d115c0a734f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.190669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] The token cache is being invalidate because project 2d4c50a599e44b2c8416385d81ed7cbb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:51.190979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.191612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.193513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 974/1975] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:51 2022] DELETE /identity/v3/projects/2d4c50a599e44b2c8416385d81ed7cbb => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:51.198007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.201723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.245856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1002/1976] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:51.256989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.258815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.282454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:51.301183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05972e66-4e7c-44e8-a6c0-d115c0a734f8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.307555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:51.308146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.308494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.309301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] PATH_INFO: `/v3/users/f3f6267abb774d209f2cf6ae05f615c6` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.310867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3f6267abb774d209f2cf6ae05f615c6)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:51.312989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.337274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243b97b8-c21b-4266-b966-65aa7148e3c1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.341967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:51.342726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.343180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.343457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.354474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:51.356077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.361787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3f6267abb774d209f2cf6ae05f615c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.365864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1003/1977] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:51.369516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3f6267abb774d209f2cf6ae05f615c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.377860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3f6267abb774d209f2cf6ae05f615c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.378528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3f6267abb774d209f2cf6ae05f615c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.392288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3f6267abb774d209f2cf6ae05f615c6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.398388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 975/1978] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:51 2022] DELETE /identity/v3/users/f3f6267abb774d209f2cf6ae05f615c6 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:51.641454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.643184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.671317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9bdd3fd-9091-4eb4-8479-7adf9f159b0f service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.701055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.703020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.730108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:51.752296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:51.753077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.753594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.753968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] PATH_INFO: `/v3/projects/e1f16d1044c54aa99da6591e604d076e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.755259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1f16d1044c54aa99da6591e604d076e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:51.757243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.787456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f4961c-e58a-43be-bdf1-05b810373e70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.794925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:51.795579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.795861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.796136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] PATH_INFO: `/v3/projects/8ffcd14987ec4e97a037be822fd7c372` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.797307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ffcd14987ec4e97a037be822fd7c372)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:51.800075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:51.828976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1f16d1044c54aa99da6591e604d076e', 'request_id': 'req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.836623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1f16d1044c54aa99da6591e604d076e', 'request_id': 'req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.847163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1f16d1044c54aa99da6591e604d076e', 'request_id': 'req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.861183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] The token cache is being invalidate because project e1f16d1044c54aa99da6591e604d076e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:51.862039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.862366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ffcd14987ec4e97a037be822fd7c372', 'request_id': 'req-1364a811-a5ac-43ab-8b09-a7cf60cf76db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.863210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.865277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1004/1979] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:51 2022] DELETE /identity/v3/projects/e1f16d1044c54aa99da6591e604d076e => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:51.871422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:51.874297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.878481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ffcd14987ec4e97a037be822fd7c372', 'request_id': 'req-1364a811-a5ac-43ab-8b09-a7cf60cf76db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.887471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ffcd14987ec4e97a037be822fd7c372', 'request_id': 'req-1364a811-a5ac-43ab-8b09-a7cf60cf76db'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.901348 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] The token cache is being invalidate because project 8ffcd14987ec4e97a037be822fd7c372 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:51.901637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.902322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1364a811-a5ac-43ab-8b09-a7cf60cf76db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:51.904015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 976/1980] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:51 2022] DELETE /identity/v3/projects/8ffcd14987ec4e97a037be822fd7c372 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:51.908523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be9ea-2615-4447-bfb1-9e9674dd8f40 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:51.909001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be9ea-2615-4447-bfb1-9e9674dd8f40 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:51.909506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607be9ea-2615-4447-bfb1-9e9674dd8f40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:51.930553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-607be9ea-2615-4447-bfb1-9e9674dd8f40 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:51.932870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-607be9ea-2615-4447-bfb1-9e9674dd8f40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:51.946617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:51.964994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a3dfe6-fed3-48cd-972f-7bb8453a6e70 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:51.994213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 977/1981] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:52.004186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041428-6e82-445b-992a-1a958918558a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.004500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041428-6e82-445b-992a-1a958918558a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.004757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02041428-6e82-445b-992a-1a958918558a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.030524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-02041428-6e82-445b-992a-1a958918558a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:52.031885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-02041428-6e82-445b-992a-1a958918558a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:52.032556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.033444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.034073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.034572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] PATH_INFO: `/v3/users/cf713100a4404af6abf04f241149441e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.035874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf713100a4404af6abf04f241149441e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.039667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.044623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 978/1982] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:19:52.045832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02041428-6e82-445b-992a-1a958918558a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.055295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02041428-6e82-445b-992a-1a958918558a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.060362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.061246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.061597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.061948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.062748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:52.063181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.064794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.079896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 979/1983] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:52.083897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.095161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f2114e-30ec-4465-8444-502d529b9028 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.101002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.102221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.102221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.102474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.102980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.105696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.108559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf713100a4404af6abf04f241149441e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.124767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf713100a4404af6abf04f241149441e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.125623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf713100a4404af6abf04f241149441e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.126376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf713100a4404af6abf04f241149441e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.136336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf713100a4404af6abf04f241149441e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.138962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 980/1984] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:52 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:52.142903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.145103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:52.150923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1005/1985] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:51 2022] DELETE /identity/v3/users/cf713100a4404af6abf04f241149441e => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:52.159595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:52.220742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac000c4-4100-4201-aaaa-81d30f016ef9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.320971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.321660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.321973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.322242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.323135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.325050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.381172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 981/1986] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:52 2022] POST /identity/v3/users => generated 286 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:52.382153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.394885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2ad5bb-ba53-4293-afdb-c327b4ddca8e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.401545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.402513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.403005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.403479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.404338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.406375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.420211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1006/1987] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:52.424808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.439978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51dd7419-31b0-4dfd-a0d7-ceb3fe4b1ecf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.450900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.452508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.452788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.453434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.454239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.456127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56e7d4f-dcef-4cc0-b4ca-f384f00fa6d2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.470762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 982/1988] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:52.475363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.490189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5301ef-601a-4be3-ad3b-f369a2bd8462 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.495636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.496317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.496595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.497149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] PATH_INFO: `/v3/projects/993ec69dc9bb4c26a199d77abcbc5811/users/d1d9f9fdb82c47ffa1aae8cf0d19a0d7/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.530024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=993ec69dc9bb4c26a199d77abcbc5811, user_id=d1d9f9fdb82c47ffa1aae8cf0d19a0d7, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.530024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.542230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1007/1989] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:52 2022] PUT /identity/v3/projects/993ec69dc9bb4c26a199d77abcbc5811/users/d1d9f9fdb82c47ffa1aae8cf0d19a0d7/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:52.548281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.548644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.548993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.571794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:52.574083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:52.622971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.624783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:52.666181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 983/1990] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:52.676322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:52.677970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:52.698880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:52.725954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:52.731891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e26cc23-ab93-4aab-bcdf-fafdebf201b7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.736769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.737491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.737750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.738015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] PATH_INFO: `/v3/projects/de3af8e838d94f5ca76305f9ef3ce955` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.739354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] RBAC: Authorizing `identity:delete_project(project_id=de3af8e838d94f5ca76305f9ef3ce955)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.742141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.750373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff67d82-c47b-4f15-bac8-efceab7963d5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:52.754608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:52.755385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.755657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.755989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.770931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:52.772491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e645d8-2b55-43de-9ea3-02000bb9f7f7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:52.785450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 984/1991] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:19:52.810313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de3af8e838d94f5ca76305f9ef3ce955', 'request_id': 'req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.814394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de3af8e838d94f5ca76305f9ef3ce955', 'request_id': 'req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.825468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de3af8e838d94f5ca76305f9ef3ce955', 'request_id': 'req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.864355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] The token cache is being invalidate because project de3af8e838d94f5ca76305f9ef3ce955 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:52.864679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.865530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9f7a3900-4208-44a5-9eb6-4df8dfa5544c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:52.873315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:52.873571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:52.873855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:52.879117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1008/1992] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:52 2022] DELETE /identity/v3/projects/de3af8e838d94f5ca76305f9ef3ce955 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:52.907966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:52.911941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:53.078943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 985/1993] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:53.086596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.086961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.087329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.127066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:53.128407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:53.158580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1009/1994] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:53.162983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.180828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f753467b-f660-4539-9391-7ab33c0a2f11 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.196501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.198461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.199225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.199225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.199840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:53.200142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.201806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.216177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 986/1995] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:53.221810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.238388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db25d59-ab9d-410d-94c3-d278492560f9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.256637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.258019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.278489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.279528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.280326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.298277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.365641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1010/1996] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/projects => generated 377 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:53.370184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.382888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32547cb1-71b2-4a75-b642-199aaeb74173 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.508018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.508804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.509950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.510366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.511132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.526624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.567752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 987/1997] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/users => generated 289 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:53.573045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.585237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27acabd-10e9-44fd-97df-62829bbc26e5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.598123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.598730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.599144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.599425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.600069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.601645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.628501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1011/1998] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:53.632558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.649725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b26aea-38d7-439d-a12f-d3a57837be2a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.672909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.673732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.674112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.675640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.676641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.678881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60dbbf92-ac72-4b4a-8679-36543900f5da admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.715104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 988/1999] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:53 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:53.716238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.731856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79762cb7-70eb-4079-a288-a82217ef44ff admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:53.739728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:53.740717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.741114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.741720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] PATH_INFO: `/v3/projects/952bc3a52bea4964ab6867f10dce9bfc/users/99e6f787a4634a708d07f54af50c5ce9/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.770137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] RBAC: Authorizing `identity:create_grant(project_id=952bc3a52bea4964ab6867f10dce9bfc, user_id=99e6f787a4634a708d07f54af50c5ce9, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:53.776086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2251c40-06fe-4554-9df1-3a5c86211d74 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:53.791106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1012/2000] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:53 2022] PUT /identity/v3/projects/952bc3a52bea4964ab6867f10dce9bfc/users/99e6f787a4634a708d07f54af50c5ce9/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:53.804026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.804382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.804846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.839935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] MFA Rules not processed for user `99e6f787a4634a708d07f54af50c5ce9`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:53.841303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:53.878510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:53.921054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:53.921346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:53.921614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:53.936562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 989/2001] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:53.941282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:53.955588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] MFA Rules not processed for user `d1d9f9fdb82c47ffa1aae8cf0d19a0d7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:53.957185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:54.017535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5162fb02-5470-498d-8050-9f99a1d0ad7a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.020427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:54.023458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.024158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.024528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.024809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.026126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.028847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.039951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1013/2002] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:54.041582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.042805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:54.058575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9525e62-eb3d-45e4-870f-6b7b881d8aff None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.061801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 990/2003] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:53 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:54.065235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.065986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.066121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.066507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.066879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.067734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.069790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.103313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1014/2004] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:54 2022] POST /identity/v3/projects => generated 371 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:54.108141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.139663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10cee2b1-adeb-4f3f-a009-9cd0f287c799 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.174603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2fd79e-7e0d-4580-8cdf-250f61be09f6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.179424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.180030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.180313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.180592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.181738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.184131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.199833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.200537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.200845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.201113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.201736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.203706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.207280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1015/2005] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:54 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:54.211161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.216419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:54.222172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1676609-13ca-465a-a0b6-95414e54a8e2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.227727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.229009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 991/2006] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:54 2022] POST /identity/v3/users => generated 295 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:54.230550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.233180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.234603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.234899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.235594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.237254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.246974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e76f8fd0-a944-4a0c-be97-18053c3b64f2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.250555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1016/2007] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:54.251728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.252335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.252609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.252883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.253651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.254709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.255272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.265502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f1cd7d-b62d-4a53-bd3f-b49b94ad67f9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.266047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 992/2008] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:54.270242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.271008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.271663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.271947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.272223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] PATH_INFO: `/v3/projects/30acd2084a644ddabcf7a0277dece983/users/7b972979cdec4efe9f29d346e23837fb/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.280560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c9c2b-f1ba-4ce0-9a63-be19644e1194 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.285323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.286119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.286529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.286705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] PATH_INFO: `/v3/projects/4264e0632605406d8d2ea2017bc9591f/users/fd7dff01f4ec4ebb9f0b357af52da81c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.288207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30acd2084a644ddabcf7a0277dece983, user_id=7b972979cdec4efe9f29d346e23837fb, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.291371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.303825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4264e0632605406d8d2ea2017bc9591f, user_id=fd7dff01f4ec4ebb9f0b357af52da81c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.305077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1017/2009] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:54 2022] PUT /identity/v3/projects/30acd2084a644ddabcf7a0277dece983/users/7b972979cdec4efe9f29d346e23837fb/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:54.307355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.309281 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.325662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 993/2010] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:54 2022] PUT /identity/v3/projects/4264e0632605406d8d2ea2017bc9591f/users/fd7dff01f4ec4ebb9f0b357af52da81c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:54.326712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.389993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d026d0d2-1bff-4935-8d98-a98ac219c355 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.408737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307017b6-b89e-4930-8153-294bc0620a4c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.413613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.414226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.414526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.414798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.415492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.417438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.425028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:54.429617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 994/2011] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:54.433963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.447885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c6c7efb-d7b9-4a59-872f-77096633372e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.464526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.465202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.465484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.466584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.467250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.468941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.473882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.474588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.474863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.475134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] PATH_INFO: `/v3/projects/4264e0632605406d8d2ea2017bc9591f/users/fd7dff01f4ec4ebb9f0b357af52da81c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.480579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4264e0632605406d8d2ea2017bc9591f, user_id=fd7dff01f4ec4ebb9f0b357af52da81c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.483492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29172868-e31c-4366-a000-3b00b3892160 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.483632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1018/2012] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:54 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:19:54.487403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.496991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 995/2013] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:54 2022] PUT /identity/v3/projects/4264e0632605406d8d2ea2017bc9591f/users/fd7dff01f4ec4ebb9f0b357af52da81c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:19:54.507363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec80084-a9dd-4be8-8cbe-cb99b86eb1db None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.507649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec80084-a9dd-4be8-8cbe-cb99b86eb1db None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.507921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fec80084-a9dd-4be8-8cbe-cb99b86eb1db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.517982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea722a3-afe5-4129-9038-cd703ff7c58b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.538735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fec80084-a9dd-4be8-8cbe-cb99b86eb1db None None] MFA Rules not processed for user `fd7dff01f4ec4ebb9f0b357af52da81c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:54.539968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fec80084-a9dd-4be8-8cbe-cb99b86eb1db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:54.590073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.590665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.590665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.590956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] PATH_INFO: `/v3/projects/30acd2084a644ddabcf7a0277dece983/users/7b972979cdec4efe9f29d346e23837fb/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.592999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] RBAC: Authorizing `identity:create_grant(project_id=30acd2084a644ddabcf7a0277dece983, user_id=7b972979cdec4efe9f29d346e23837fb, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.595778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.608438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1019/2014] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:54 2022] PUT /identity/v3/projects/30acd2084a644ddabcf7a0277dece983/users/7b972979cdec4efe9f29d346e23837fb/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:54.640507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 996/2015] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:54 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:54.648681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:54.650693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:54.723577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a253cddf-a4f1-477d-b05c-984f0d7a382f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:54.729014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:54.729879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:54.730167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:54.730521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:54.812173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:54.813908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:54.825124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1020/2016] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:54 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:55.380278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:55.380787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:55.380986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:55.419161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] MFA Rules not processed for user `7b972979cdec4efe9f29d346e23837fb`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:55.420480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:55.440469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:55.499172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 997/2017] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:55 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:55.510997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:55.512490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:55.524524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ffea19-ec8c-4889-bae3-9628e5666eb4 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:55.529672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:55.530325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:55.530609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:55.530881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:55.553113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:55.604652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:55.605956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:55.617146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1021/2018] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:57.045378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:57.055360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f18a3a7d-c864-4dca-8ff0-d4b7b6399e1f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:57.059595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:57.060337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:57.060609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:57.060878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:57.070739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:57.073340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7380e9e3-09dd-4274-b80a-1dff2db0119e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:57.084716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 998/2019] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:57 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:57.959464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:57.960977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.028314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71d1ebb-09fd-4204-bdbe-5df6e3daa67b service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:58.033022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:58.033666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.033941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.034212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] PATH_INFO: `/v3/users/a0caac57a68a4262880ad77475e6b7ff` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.035385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0caac57a68a4262880ad77475e6b7ff)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:58.037351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:58.088363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0caac57a68a4262880ad77475e6b7ff'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.100056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0caac57a68a4262880ad77475e6b7ff'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.100722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0caac57a68a4262880ad77475e6b7ff'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.101340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0caac57a68a4262880ad77475e6b7ff'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.109201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0caac57a68a4262880ad77475e6b7ff'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.122199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1022/2020] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:19:57 2022] DELETE /identity/v3/users/a0caac57a68a4262880ad77475e6b7ff => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:58.126811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.127286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.127378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.146562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:58.147771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.207046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 999/2021] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:58.217455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:58.218926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.264781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:58.285083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870f8a9-6e03-4d2c-9a3f-9aed86271f88 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:58.288699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:58.289311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.289587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.289858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.299608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:58.301018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df635187-09cc-4090-ab02-07144ca81a2e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:58.311417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1023/2022] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:19:58.692033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:58.694278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.721880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c70804-f852-466c-af53-3a3325e2a775 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:58.741170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:58.741170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.742101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.742101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] PATH_INFO: `/v3/projects/de117a6ecbae4791a29c165fc99d9089` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.745320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=de117a6ecbae4791a29c165fc99d9089)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:58.747377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:58.798865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'de117a6ecbae4791a29c165fc99d9089', 'request_id': 'req-ee55856a-e885-4be4-99a7-e939a5f9ecf4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.806823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'de117a6ecbae4791a29c165fc99d9089', 'request_id': 'req-ee55856a-e885-4be4-99a7-e939a5f9ecf4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.813858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'de117a6ecbae4791a29c165fc99d9089', 'request_id': 'req-ee55856a-e885-4be4-99a7-e939a5f9ecf4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.825295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] The token cache is being invalidate because project de117a6ecbae4791a29c165fc99d9089 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:19:58.825613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.826315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ee55856a-e885-4be4-99a7-e939a5f9ecf4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:19:58.827835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1000/2023] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:19:58 2022] DELETE /identity/v3/projects/de117a6ecbae4791a29c165fc99d9089 => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:58.832833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.833144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.833420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.854109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:58.855339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.914267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1024/2024] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:58.922547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.922885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.923165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.944745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:58.945993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:58.958522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1001/2025] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:58.963335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:58.972950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1cbed1-c32d-4bca-9140-6446c377d9aa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:58.977642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:58.979048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:58.979468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:58.979746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:58.980412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:19:58.980723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:58.982375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:58.995402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1025/2026] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:19:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:19:58.999748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.008957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1438d6-9d97-4fee-8b32-a105f0996fe7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.013641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.014233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.014614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.014886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.015511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.017699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.032885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:59.047325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1002/2027] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:19:58 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:19:59.051507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.062494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1b26-5cd3-42fd-a54b-0df9a79d7aae admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.128147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.128766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.129048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.129321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.130005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.131976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.157463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1026/2028] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:19:59 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:19:59.161417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.173131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07128f2a-f00d-4943-8d01-b0a076533e90 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.177842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.178469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.178748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.179018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.179788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.181583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.192128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1003/2029] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:59.197268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.207226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e0cf36e-a2bd-4090-9b5b-a32231140651 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.211858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.213099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.213330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.213647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.214381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.215975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70f385-9638-4ccd-97ce-bc703a3c09c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.227621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1027/2030] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:19:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:19:59.235454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.246727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6bf4c5e-3013-436b-b635-2ee4efb40fe5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.251836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.252493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.252769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.253043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] PATH_INFO: `/v3/projects/e0af279e53714aebbb1b15c9d7d48757/users/09020c01783d4c9f9a5acb532a77693d/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.268645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0af279e53714aebbb1b15c9d7d48757, user_id=09020c01783d4c9f9a5acb532a77693d, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.271325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.283763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1004/2031] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:19:59 2022] PUT /identity/v3/projects/e0af279e53714aebbb1b15c9d7d48757/users/09020c01783d4c9f9a5acb532a77693d/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:19:59.294112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.294425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.294695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.322374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] MFA Rules not processed for user `09020c01783d4c9f9a5acb532a77693d`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:19:59.323721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:59.354650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:19:59.389224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1028/2032] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:19:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:19:59.400669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:19:59.402484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:19:59.462080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7ab77b-a81f-4ae0-b65f-cadce91027f5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:19:59.466165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:19:59.466802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:19:59.467080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:19:59.467351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:19:59.476061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:19:59.477500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:19:59.486843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1005/2033] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:19:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:01.771095 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:01.781640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f870ab83-4100-4c91-9d9b-9d9b57220bc9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:01.785214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:01.785813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:01.786091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:01.786392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:01.788155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:01.860347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:01.861937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26aed85a-cd8c-4c51-8d4e-4d537d1f6c08 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:01.872126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1029/2034] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:01 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:08.064668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:08.066127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:08.152496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-803166ba-79ae-4b3e-9128-61b635941d8b service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:08.158289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:08.159022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:08.159358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:08.159677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] PATH_INFO: `/v3/users/5f5875db99024e58a7c8b5d3009aefb0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:08.160910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] RBAC: Authorizing `identity:delete_user(user_id=5f5875db99024e58a7c8b5d3009aefb0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:08.163044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:08.219087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5f5875db99024e58a7c8b5d3009aefb0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.227161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5f5875db99024e58a7c8b5d3009aefb0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.239872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5f5875db99024e58a7c8b5d3009aefb0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.240861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5f5875db99024e58a7c8b5d3009aefb0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.243207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:08.256889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5f5875db99024e58a7c8b5d3009aefb0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.263603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1006/2035] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:08 2022] DELETE /identity/v3/users/5f5875db99024e58a7c8b5d3009aefb0 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:08.268499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:08.268947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:08.269041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:08.289599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:08.291327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:08.351985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1030/2036] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:08.362488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:08.364352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:08.436236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d32ff-a58c-4eeb-ac30-c3e8bd29243d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:08.440267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:08.441979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:08.442277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:08.442564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:08.454418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:08.456629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7281289-19c0-41f6-9df6-996e2db9ef5d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:08.468076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1007/2037] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:08 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:08.812642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:08.814738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:08.832787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cd20a9-df94-4343-a351-a87707a8708d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:08.850279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:08.853974 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:08.854574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:08.854997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] PATH_INFO: `/v3/projects/15aa5430422e4d0f9181884b58742b15` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:08.858040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] RBAC: Authorizing `identity:delete_project(project_id=15aa5430422e4d0f9181884b58742b15)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:08.861250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:08.927685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15aa5430422e4d0f9181884b58742b15', 'request_id': 'req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.934599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15aa5430422e4d0f9181884b58742b15', 'request_id': 'req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.942606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15aa5430422e4d0f9181884b58742b15', 'request_id': 'req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.945185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:08.957380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] The token cache is being invalidate because project 15aa5430422e4d0f9181884b58742b15 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:08.957712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.958456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4bb99c1e-6f2c-45fb-9393-dca0e0731fea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:08.960017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1031/2038] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:08 2022] DELETE /identity/v3/projects/15aa5430422e4d0f9181884b58742b15 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:08.965976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:08.966387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:08.966725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:08.986671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:08.987893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.050699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1008/2039] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:08 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:09.058938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.059328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.059665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.080456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:09.081896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.093785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1032/2040] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:09.095015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.105860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43c64b2f-a3d0-477b-820e-f505132c0066 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.112871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.118520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.118901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.119242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.119960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:09.120328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.121998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.135509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1009/2041] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:09.139926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.150616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e9c57a-3588-4625-beea-5233fb1cd18a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.155703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.156303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.156646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.156925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.157560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.159474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.193951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1033/2042] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:09 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:09.195276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.206427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d375aa2-627b-4c02-881c-19a630d8e165 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.266844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.267747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.268025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.268294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.268924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.270854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.293282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1010/2043] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:09 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:09.297333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.308097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e152a52-f3c5-4453-894f-60335f754365 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.312584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.313177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.313455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.313725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.314401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.315984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.326300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1034/2044] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:09.330211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.335366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:09.343135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2046dfc5-a786-4fab-b4ef-f558a67ce44c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.348828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.349424 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.349698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.349963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.350662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.352252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a78179-bf78-433a-8814-37d1a861c86d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.363152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1011/2045] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:09 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:09.367706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.377854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f44d50-7f3a-437f-8f3b-c4460d7f2a07 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.382652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.383310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.383581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.383851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] PATH_INFO: `/v3/projects/4d60afc13ff44b55a30220bfb98942a9/users/e381d88ba52744d9af4a03287be57281/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.399023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4d60afc13ff44b55a30220bfb98942a9, user_id=e381d88ba52744d9af4a03287be57281, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.402372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.414846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1035/2046] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:09 2022] PUT /identity/v3/projects/4d60afc13ff44b55a30220bfb98942a9/users/e381d88ba52744d9af4a03287be57281/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:09.426273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.426599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.426871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.454881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] MFA Rules not processed for user `e381d88ba52744d9af4a03287be57281`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:09.456088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.518477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1012/2047] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:09 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:09.529787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.531810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.593690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef52b7f3-816c-4892-8cc3-6b7c4e79be7f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.597690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.598330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.598610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.599000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.608074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.609624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130fc0e6-184f-485a-a908-6a8c1e4982ef service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.621852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1036/2048] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:09 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:09.646811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.709142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038d95c0-4ef8-475e-9093-0dde9b511e57 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.714616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.715271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.715575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.715848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] PATH_INFO: `/v3/users/e381d88ba52744d9af4a03287be57281` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.717052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e381d88ba52744d9af4a03287be57281)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.719050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.766114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e381d88ba52744d9af4a03287be57281'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:09.773291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e381d88ba52744d9af4a03287be57281'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:09.781092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e381d88ba52744d9af4a03287be57281'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:09.781907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e381d88ba52744d9af4a03287be57281'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:09.794470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e381d88ba52744d9af4a03287be57281'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:09.796596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:09.802234 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1013/2049] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:09 2022] DELETE /identity/v3/users/e381d88ba52744d9af4a03287be57281 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:09.807135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.807422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.807699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.828232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:09.829436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.888366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1037/2050] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:09.898644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:09.900718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:09.969616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a8d3c6-9e39-43a5-ae34-6f915c3c037f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:09.973437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:09.974082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:09.974394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:09.974666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:09.987015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:09.988460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898f010-da35-41a6-a523-0e2453d6d02e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:09.998944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1014/2051] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:10.319295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.320558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:10.333491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c81d60d-b815-427d-9798-b9ac1fdcd621 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.339071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.340026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.340457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.341290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] PATH_INFO: `/v3/projects/4d60afc13ff44b55a30220bfb98942a9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.342698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4d60afc13ff44b55a30220bfb98942a9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.344819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.380671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:10.400768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4d60afc13ff44b55a30220bfb98942a9', 'request_id': 'req-e9c29bb4-e00e-4c7a-8214-d62197fef394'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:10.407402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4d60afc13ff44b55a30220bfb98942a9', 'request_id': 'req-e9c29bb4-e00e-4c7a-8214-d62197fef394'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:10.416795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4d60afc13ff44b55a30220bfb98942a9', 'request_id': 'req-e9c29bb4-e00e-4c7a-8214-d62197fef394'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:10.429067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] The token cache is being invalidate because project 4d60afc13ff44b55a30220bfb98942a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:10.429348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:10.429989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e9c29bb4-e00e-4c7a-8214-d62197fef394 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:10.432349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1038/2052] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:10 2022] DELETE /identity/v3/projects/4d60afc13ff44b55a30220bfb98942a9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:10.437095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.437600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.437850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.458833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:10.460150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:10.520963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1015/2053] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:10.529702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.529990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.530278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.550495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:10.551709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:10.560469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1039/2054] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:10.565636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.575974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979d353c-9cf0-4a81-baf3-d5ebc2e48678 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.580469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.581054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.581327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.581595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.582236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:10.582557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.584156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.599573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1016/2055] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:10.600598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.610112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0996fe8d-b835-450e-9317-61966ea32973 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.614834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.615435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.615717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.615988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.616696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.619248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.655835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1040/2056] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:10 2022] POST /identity/v3/projects => generated 373 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:10.660121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.671021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bfbd80a-6313-40d0-afeb-0576e19614b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.728542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.729149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.729441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.729693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.730396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.732307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.756855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1017/2057] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:10 2022] POST /identity/v3/users => generated 287 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:10.758076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.770743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50eb705-a422-4c59-9b85-36b1490a3bbd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.775586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.776187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.776465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.776733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.777373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.779000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.788840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1041/2058] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:10.793574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.803357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4214595f-4d61-475e-abdd-0b0b66b7e8fc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.808567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.809160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.809435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.809701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.810364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.811953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6c8000e-b5b5-4463-a989-ad45bdfb9cc3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.823575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1018/2059] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:10 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:10.827716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.837711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0da55e6-98e1-4a25-9ede-12c923c148af admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:10.842323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:10.842969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.843248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.843518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] PATH_INFO: `/v3/projects/bcbcbb829ba145c3981a4fe4257fb003/users/1ace2165f11e4f2eb9c3bf17d27ee370/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.859197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bcbcbb829ba145c3981a4fe4257fb003, user_id=1ace2165f11e4f2eb9c3bf17d27ee370, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:10.861835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:10.875192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1042/2060] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:10 2022] PUT /identity/v3/projects/bcbcbb829ba145c3981a4fe4257fb003/users/1ace2165f11e4f2eb9c3bf17d27ee370/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:10.886220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:10.886562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:10.886862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:10.907044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:10.917075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] MFA Rules not processed for user `1ace2165f11e4f2eb9c3bf17d27ee370`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:10.918335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:10.982127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1019/2061] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:10.992719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:10.994928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:11.050790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a166b02f-eb6c-4b6c-bb52-e0c8f874d731 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:11.054394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:11.055349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:11.055349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:11.055602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:11.063662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:11.065092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:11.074927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1043/2062] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:10 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:20.979233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:21.069834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=6,keystone:SELECT=67,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:24.206697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:24.208178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:24.275971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865068cd-9422-4cd9-8036-195e897fc612 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:24.280474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:24.281031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:24.281305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:24.281571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] PATH_INFO: `/v3/users/d1d9f9fdb82c47ffa1aae8cf0d19a0d7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:24.282720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1d9f9fdb82c47ffa1aae8cf0d19a0d7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:24.284647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:24.328941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1d9f9fdb82c47ffa1aae8cf0d19a0d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.335601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1d9f9fdb82c47ffa1aae8cf0d19a0d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.343589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1d9f9fdb82c47ffa1aae8cf0d19a0d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.344208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1d9f9fdb82c47ffa1aae8cf0d19a0d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.356762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1d9f9fdb82c47ffa1aae8cf0d19a0d7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.362584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1020/2063] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:24 2022] DELETE /identity/v3/users/d1d9f9fdb82c47ffa1aae8cf0d19a0d7 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:24.534385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:24.549219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a052db-628f-4806-9c56-fe5fd20db4af service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:24.606962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:24.607587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:24.607865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:24.608133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] PATH_INFO: `/v3/projects/993ec69dc9bb4c26a199d77abcbc5811` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:24.609236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] RBAC: Authorizing `identity:delete_project(project_id=993ec69dc9bb4c26a199d77abcbc5811)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:24.611168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:24.664017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '993ec69dc9bb4c26a199d77abcbc5811', 'request_id': 'req-53374a60-fc84-4d44-8ef3-762caae18253'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.671021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '993ec69dc9bb4c26a199d77abcbc5811', 'request_id': 'req-53374a60-fc84-4d44-8ef3-762caae18253'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.681469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '993ec69dc9bb4c26a199d77abcbc5811', 'request_id': 'req-53374a60-fc84-4d44-8ef3-762caae18253'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.692537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] The token cache is being invalidate because project 993ec69dc9bb4c26a199d77abcbc5811 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:24.692829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.693508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.695563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1044/2064] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:24 2022] DELETE /identity/v3/projects/993ec69dc9bb4c26a199d77abcbc5811 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:24.700155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:24.701770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:24.716154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fcbcd-8314-4d96-bd9e-0d6815fac641 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:24.776054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:24.776819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:24.777107 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:24.777384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] PATH_INFO: `/v3/users/7b972979cdec4efe9f29d346e23837fb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:24.778579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7b972979cdec4efe9f29d346e23837fb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:24.780516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:24.805373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:24.848177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7b972979cdec4efe9f29d346e23837fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.856009 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7b972979cdec4efe9f29d346e23837fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.863508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7b972979cdec4efe9f29d346e23837fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.864200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7b972979cdec4efe9f29d346e23837fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.877175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53dd7a70-3fa0-45d9-a614-1470afe2faa7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7b972979cdec4efe9f29d346e23837fb'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:24.884021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1021/2065] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:24 2022] DELETE /identity/v3/users/7b972979cdec4efe9f29d346e23837fb => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:25.057434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.058945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:25.081545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53374a60-fc84-4d44-8ef3-762caae18253 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.141784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.142710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.142984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.143250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] PATH_INFO: `/v3/projects/30acd2084a644ddabcf7a0277dece983` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.144601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] RBAC: Authorizing `identity:delete_project(project_id=30acd2084a644ddabcf7a0277dece983)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.146513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.184264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:25.198334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30acd2084a644ddabcf7a0277dece983', 'request_id': 'req-fed02de7-9925-49a6-8dc8-4157eef1d80d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:25.204496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30acd2084a644ddabcf7a0277dece983', 'request_id': 'req-fed02de7-9925-49a6-8dc8-4157eef1d80d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:25.212490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30acd2084a644ddabcf7a0277dece983', 'request_id': 'req-fed02de7-9925-49a6-8dc8-4157eef1d80d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:25.223138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] The token cache is being invalidate because project 30acd2084a644ddabcf7a0277dece983 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:25.223421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:25.224551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fed02de7-9925-49a6-8dc8-4157eef1d80d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:25.226091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1045/2066] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:25 2022] DELETE /identity/v3/projects/30acd2084a644ddabcf7a0277dece983 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:25.232500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.233074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.233074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.253474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:25.254726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:25.315294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1022/2067] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:25.324229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.324514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.324788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.345461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:25.346686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:25.355882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1046/2068] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:25.360181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.370980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5a7dc9-ba5b-4e03-832b-f59fd1c4ac32 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.375587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.376185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.376464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.376734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.377590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:25.377894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.379823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.393316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1023/2069] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:25.397868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.410713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd6a568-3f08-4b80-85a4-b7726a23aebb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.416330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.416964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.417267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.417964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.418210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.420184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.449297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1047/2070] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:25.453552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.463954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe2654a-e018-4235-abfc-428f25fe188d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.521080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.521691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.521971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.522268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.522912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.524992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.547267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1024/2071] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:25.553209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.568987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b510317d-2d05-4736-8f13-c409ef2389fe admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.576322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.577038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.577414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.577786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.578582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.580265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.590942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1048/2072] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:25.595233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.605790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1252ef7-fffa-4ae4-833d-8e7c1928edef admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.610871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.611476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.611756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.612274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.613165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.614526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c9241d-521c-4523-b7bc-bd162da36827 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.626197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1025/2073] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:25 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:25.630182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.640763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054cb81f-0b0c-4fb5-805c-6b29b643dbc2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.645648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.646318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.646605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.646875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] PATH_INFO: `/v3/projects/68e6f0d5b8ae45faaccd4d665d683bd4/users/78c2f7d45fda41569d376f98316613f5/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.661532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] RBAC: Authorizing `identity:create_grant(project_id=68e6f0d5b8ae45faaccd4d665d683bd4, user_id=78c2f7d45fda41569d376f98316613f5, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.664191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.676792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1049/2074] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:25 2022] PUT /identity/v3/projects/68e6f0d5b8ae45faaccd4d665d683bd4/users/78c2f7d45fda41569d376f98316613f5/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:25.687812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.688098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.688396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.697623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:25.718707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] MFA Rules not processed for user `78c2f7d45fda41569d376f98316613f5`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:25.719922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:25.786376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1026/2075] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:25.791622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.866962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14e973d2-b10d-406b-9f1d-ac540905d933 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.872785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.872785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.873016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.873258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.873881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.875787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:25.906936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1050/2076] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/projects => generated 383 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:25.908024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:25.918871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2657aa-469f-43b6-97a5-e26dd8bcd1cf None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:25.979149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:25.979760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:25.980041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:25.980316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:25.980965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:25.983011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.006312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1027/2077] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:25 2022] POST /identity/v3/users => generated 298 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:26.009278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:26.023889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c3df26-c9fb-4d72-b698-64160c6b2ed9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:26.030276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:26.030898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.031179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.031453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.032096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:26.033714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.048436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1051/2078] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:26.049406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:26.059485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1aaa629-6eaa-4a4b-8e49-b5102fc43ea2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:26.064050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:26.064691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.064966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.065239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] PATH_INFO: `/v3/projects/66cd20831575476fb271efef898cc23f/users/f8ff1a22d96844aca6e914650e7ee396/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.080768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cd20831575476fb271efef898cc23f, user_id=f8ff1a22d96844aca6e914650e7ee396, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:26.083499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.097267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1028/2079] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:26 2022] PUT /identity/v3/projects/66cd20831575476fb271efef898cc23f/users/f8ff1a22d96844aca6e914650e7ee396/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:26.101424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:26.111735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3436ab2b-f928-44e0-9a45-2eaf1ea7a805 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:26.141898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:26.173089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:26.173699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.173975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.174242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.174904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:26.176616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c2d64b-ec24-4265-86bd-8c353871b872 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.187474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1052/2080] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:26 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:26.191635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:26.202586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1369db-d90b-449c-9e02-3f0426db7764 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:26.207220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:26.207863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.208142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.208413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] PATH_INFO: `/v3/projects/66cd20831575476fb271efef898cc23f/users/f8ff1a22d96844aca6e914650e7ee396/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.210313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=66cd20831575476fb271efef898cc23f, user_id=f8ff1a22d96844aca6e914650e7ee396, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:26.213677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.226538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1029/2081] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:26 2022] PUT /identity/v3/projects/66cd20831575476fb271efef898cc23f/users/f8ff1a22d96844aca6e914650e7ee396/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:26.239944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.240254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.240527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.268410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] MFA Rules not processed for user `f8ff1a22d96844aca6e914650e7ee396`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:26.269608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:26.341580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1053/2082] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:26 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:26.352656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:26.354449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:26.409543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:26.415767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a793bc8-0a4c-4141-a9ec-70993439e9e9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:26.419544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:26.420142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:26.420415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:26.420679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:26.478392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:26.479834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcddc782-bf27-47d6-8f19-dfac6df63044 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:26.488978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1030/2083] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:28.265247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:28.266730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:28.425711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c816499-9c7d-4c03-aa73-290d83fff501 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:28.444221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:28.445384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:28.445717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:28.445992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] PATH_INFO: `/v3/users/1ace2165f11e4f2eb9c3bf17d27ee370` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:28.447252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ace2165f11e4f2eb9c3bf17d27ee370)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:28.449231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:28.527203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ace2165f11e4f2eb9c3bf17d27ee370'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:28.538179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:28.550755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ace2165f11e4f2eb9c3bf17d27ee370'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:28.551533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ace2165f11e4f2eb9c3bf17d27ee370'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:28.552215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ace2165f11e4f2eb9c3bf17d27ee370'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:28.563612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ace2165f11e4f2eb9c3bf17d27ee370'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:28.583935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1054/2084] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:28 2022] DELETE /identity/v3/users/1ace2165f11e4f2eb9c3bf17d27ee370 => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:28.587155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:28.589623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:28.590463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:28.619164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:28.621572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:28.726349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1031/2085] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:28.733741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:28.735277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:28.817600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776c4b3-caac-408d-8322-d87455dfbf8c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:28.824062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:28.825117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:28.825587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:28.826040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:28.841533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:28.844114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:28.863273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1055/2086] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:29.293822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:29.295415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:29.311973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e85ef51-16c5-4248-8224-f458f94a89d7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:29.318336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:29.319018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:29.319417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:29.319771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] PATH_INFO: `/v3/projects/bcbcbb829ba145c3981a4fe4257fb003` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:29.321006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bcbcbb829ba145c3981a4fe4257fb003)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:29.323693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:29.404458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bcbcbb829ba145c3981a4fe4257fb003', 'request_id': 'req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:29.416129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bcbcbb829ba145c3981a4fe4257fb003', 'request_id': 'req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:29.427786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bcbcbb829ba145c3981a4fe4257fb003', 'request_id': 'req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:29.442850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] The token cache is being invalidate because project bcbcbb829ba145c3981a4fe4257fb003 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:29.443228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:29.444020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:29.454400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1032/2087] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:29 2022] DELETE /identity/v3/projects/bcbcbb829ba145c3981a4fe4257fb003 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:31.166949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:31.169213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.191655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:31.193877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.249460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:31.294999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0535db-79f9-4a8e-a0f8-bfe63bdb4b92 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:31.308979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:31.310763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:31.311019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:31.311297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] PATH_INFO: `/v3/users/99e6f787a4634a708d07f54af50c5ce9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:31.312933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=99e6f787a4634a708d07f54af50c5ce9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:31.322414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:31.355447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4cc275-7c81-476b-9c25-048a5feb969b service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:31.442887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99e6f787a4634a708d07f54af50c5ce9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.456704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:31.457526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:31.457990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:31.458439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:31.458572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99e6f787a4634a708d07f54af50c5ce9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.460695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.472426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99e6f787a4634a708d07f54af50c5ce9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.473414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99e6f787a4634a708d07f54af50c5ce9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.499195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8582c8cc-bd2d-4b0f-9deb-23d927622a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99e6f787a4634a708d07f54af50c5ce9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.511822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1033/2088] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:31 2022] DELETE /identity/v3/users/99e6f787a4634a708d07f54af50c5ce9 => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:31.518499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:31.518499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:31.518499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:31.542351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:31.543512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.551209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:31.587527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:31.591490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:31.617382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1056/2089] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:31 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:31.637932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1034/2090] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:31 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:31.650218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:31.653066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.663755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:31.665453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.680245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84df26e-f603-4e7e-b1d7-9e7b50b2e356 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:31.687843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:31.688492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:31.688802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:31.689204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] PATH_INFO: `/v3/users/b0e68c885560485ead1c61fe83fc3ce4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:31.690442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0e68c885560485ead1c61fe83fc3ce4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:31.692749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:31.744594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72ebb717-889f-4743-bc6f-7da8cd46e272 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:31.752363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:31.770875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0e68c885560485ead1c61fe83fc3ce4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.783058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0e68c885560485ead1c61fe83fc3ce4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.794969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0e68c885560485ead1c61fe83fc3ce4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.795601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0e68c885560485ead1c61fe83fc3ce4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.825670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0e68c885560485ead1c61fe83fc3ce4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:31.831526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:31.832138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:31.832417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:31.832685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:31.833926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1035/2091] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:31 2022] DELETE /identity/v3/users/b0e68c885560485ead1c61fe83fc3ce4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:31.836544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:31.859837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:31.862872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:31.876411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1057/2092] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:31 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:32.136830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.138753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.267964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f27269-f1c3-4f30-95fd-d5871d4868bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.276121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.276555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.277018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.277441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] PATH_INFO: `/v3/projects/fe278e07fd7940dca53f7c2b8b792326` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.279178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe278e07fd7940dca53f7c2b8b792326)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.281186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.345383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.347274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.360857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9008d6-eca6-4a94-a264-f4b13b31da53 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.380819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe278e07fd7940dca53f7c2b8b792326', 'request_id': 'req-4b17654e-12a3-4296-8974-4f86c8faeee8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.390968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe278e07fd7940dca53f7c2b8b792326', 'request_id': 'req-4b17654e-12a3-4296-8974-4f86c8faeee8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.403164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe278e07fd7940dca53f7c2b8b792326', 'request_id': 'req-4b17654e-12a3-4296-8974-4f86c8faeee8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.416389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] The token cache is being invalidate because project fe278e07fd7940dca53f7c2b8b792326 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:32.416749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.417425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4b17654e-12a3-4296-8974-4f86c8faeee8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.419010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1036/2093] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:32 2022] DELETE /identity/v3/projects/fe278e07fd7940dca53f7c2b8b792326 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:32.423626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80193-4f2a-4575-b6ae-1894d4151009 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.423907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80193-4f2a-4575-b6ae-1894d4151009 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.424179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e80193-4f2a-4575-b6ae-1894d4151009 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.433483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.434241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.434618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.434956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] PATH_INFO: `/v3/projects/952bc3a52bea4964ab6867f10dce9bfc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.436160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] RBAC: Authorizing `identity:delete_project(project_id=952bc3a52bea4964ab6867f10dce9bfc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.439512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.449263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a9e80193-4f2a-4575-b6ae-1894d4151009 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:32.450507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a9e80193-4f2a-4575-b6ae-1894d4151009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.458719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1037/2094] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:32.466711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.467061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.467402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.491672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:32.494024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.498648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:32.505957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '952bc3a52bea4964ab6867f10dce9bfc', 'request_id': 'req-c8494aa7-9c3e-48b9-9375-d7aad5880093'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.514621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '952bc3a52bea4964ab6867f10dce9bfc', 'request_id': 'req-c8494aa7-9c3e-48b9-9375-d7aad5880093'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.523991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '952bc3a52bea4964ab6867f10dce9bfc', 'request_id': 'req-c8494aa7-9c3e-48b9-9375-d7aad5880093'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.531457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:32.543421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] The token cache is being invalidate because project 952bc3a52bea4964ab6867f10dce9bfc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:32.543945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.545014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.546933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1058/2095] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:32 2022] DELETE /identity/v3/projects/952bc3a52bea4964ab6867f10dce9bfc => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:32.552163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.554388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.560620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1038/2096] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:32.566769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.572830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8494aa7-9c3e-48b9-9375-d7aad5880093 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.577832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.578482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.578826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.579116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] PATH_INFO: `/v3/users/fd7dff01f4ec4ebb9f0b357af52da81c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.580300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd7dff01f4ec4ebb9f0b357af52da81c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.580557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d1283c-d96a-4643-8492-e10079c8909b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.582839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.589025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.589533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.589820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.590096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.590796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:32.591108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.592752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.606699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1039/2097] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:20:32.611311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.627075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544955b6-5fdb-4597-82e8-e09dbf339170 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.632708 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.633350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.634280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.634578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.635340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.637332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.643849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd7dff01f4ec4ebb9f0b357af52da81c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.657288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd7dff01f4ec4ebb9f0b357af52da81c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.658039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd7dff01f4ec4ebb9f0b357af52da81c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.658716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd7dff01f4ec4ebb9f0b357af52da81c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.666482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd7dff01f4ec4ebb9f0b357af52da81c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:32.667889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1040/2098] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:32 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:32.671937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.673535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.682190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1059/2099] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:32 2022] DELETE /identity/v3/users/fd7dff01f4ec4ebb9f0b357af52da81c => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:32.744287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6221bd25-3181-4003-9e6a-45223cccc893 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.803846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.804502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.804779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.805043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.806887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.808979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.839132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1041/2100] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:32 2022] POST /identity/v3/users => generated 283 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:32.844982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.867690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf3b7e4-d701-4e45-8588-a0f943753ef7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.876322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.876995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.877268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.877532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.878818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.880480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.892843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1060/2101] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:32.896781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.907928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb42683-d4b3-4707-b8da-4f6c292af768 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.912734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.913407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.913750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.914109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.914869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.916456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.926815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1042/2102] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:32.932071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.943076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84963bc6-4ab4-40ce-90b1-72095ded96e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:32.948685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:32.949448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:32.949745 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:32.950000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] PATH_INFO: `/v3/projects/a6709c79943049a18d7190f0a98b8d7b/users/b4726382ddb046b7965745189808bc43/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:32.965193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6709c79943049a18d7190f0a98b8d7b, user_id=b4726382ddb046b7965745189808bc43, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:32.967881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae37c05c-c295-4696-960f-1fccb7153a91 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:32.980771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:32.983397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:32.990338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1061/2103] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:32 2022] PUT /identity/v3/projects/a6709c79943049a18d7190f0a98b8d7b/users/b4726382ddb046b7965745189808bc43/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:32.999040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c532e72-5e1d-4d4c-b370-dd403f90337e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.004093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.004403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.004672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.035561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] MFA Rules not processed for user `b4726382ddb046b7965745189808bc43`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:33.036787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.047704 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:33.103589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.104250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.104562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.104956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] PATH_INFO: `/v3/projects/4264e0632605406d8d2ea2017bc9591f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.106207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4264e0632605406d8d2ea2017bc9591f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.109039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.115868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:33.130830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1062/2104] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:33.133339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.163305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9569d7-088d-415f-bf38-6357df02f0d9 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.178568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4264e0632605406d8d2ea2017bc9591f', 'request_id': 'req-a5d72ee4-825f-494a-86b1-c077d7ed4b18'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:33.185402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4264e0632605406d8d2ea2017bc9591f', 'request_id': 'req-a5d72ee4-825f-494a-86b1-c077d7ed4b18'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:33.193373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4264e0632605406d8d2ea2017bc9591f', 'request_id': 'req-a5d72ee4-825f-494a-86b1-c077d7ed4b18'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:33.213479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] The token cache is being invalidate because project 4264e0632605406d8d2ea2017bc9591f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:33.213840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:33.214553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a5d72ee4-825f-494a-86b1-c077d7ed4b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:33.216254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1043/2105] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:32 2022] DELETE /identity/v3/projects/4264e0632605406d8d2ea2017bc9591f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:33.220755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025257df-8f6f-4b26-89e2-e8b4a5cdd917 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.221041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025257df-8f6f-4b26-89e2-e8b4a5cdd917 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.221319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025257df-8f6f-4b26-89e2-e8b4a5cdd917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.242810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-025257df-8f6f-4b26-89e2-e8b4a5cdd917 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:33.244566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-025257df-8f6f-4b26-89e2-e8b4a5cdd917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.260014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.260708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.261057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.261399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.262108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.264096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.297373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1063/2106] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/projects => generated 363 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:33.301456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.303665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.333330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1044/2107] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:33.342852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.343194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.343472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.373192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:33.374974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.385010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1045/2108] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:33.387320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d075b656-0924-43ea-8162-8dedfde7de19 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.389180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.393537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.394191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.397944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.398236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.399740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.400739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9becc9b5-4faf-4cf1-8465-b6359b4c50bc None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.402398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.406401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.407015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.407377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.407564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.408219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:33.408523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.410968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.426985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1046/2109] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:20:33.427324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1064/2110] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/users => generated 288 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:33.434088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.434924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.445454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63686eb5-444a-4685-a953-c959f4564a98 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.445666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5052d0e8-7575-453e-b089-ee4872a65b9f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.451630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.452299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.452657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.452766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.453012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.453451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.453794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.453887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.454122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.454789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.455687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.456900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.468176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1047/2111] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:33.472299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.485168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f5718b-1916-417b-8937-afeb3b120607 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.490701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.491405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.491678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.491943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] PATH_INFO: `/v3/projects/851c6adc6a38474daed4ba29127f618d/users/b5843ea9864340cebf453f4b348811a0/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.492219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1065/2112] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:33.497579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.513719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=851c6adc6a38474daed4ba29127f618d, user_id=b5843ea9864340cebf453f4b348811a0, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.516851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.533457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1048/2113] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:33 2022] PUT /identity/v3/projects/851c6adc6a38474daed4ba29127f618d/users/b5843ea9864340cebf453f4b348811a0/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:33.537479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.551812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:33.594263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:33.613197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd4f09-1eb9-45e3-82e2-004ca58796e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.644677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab923bf9-f195-4d0b-a777-7638bf7cb7e1 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.650776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.651897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.652258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.652597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.653328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.655128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.667764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1049/2114] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:33.672079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.683348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.683880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05e469d-09d0-40ea-a396-cf367cdaa16a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.684004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.684267 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.684684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.685350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.687339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6bc198c-833d-4785-9c7b-4f9e0cbad48d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.689230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.689883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.690184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.690470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] PATH_INFO: `/v3/projects/851c6adc6a38474daed4ba29127f618d/users/b5843ea9864340cebf453f4b348811a0/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.692594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] RBAC: Authorizing `identity:create_grant(project_id=851c6adc6a38474daed4ba29127f618d, user_id=b5843ea9864340cebf453f4b348811a0, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.695381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.707871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1050/2115] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:33 2022] PUT /identity/v3/projects/851c6adc6a38474daed4ba29127f618d/users/b5843ea9864340cebf453f4b348811a0/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:33.713493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1066/2116] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/users => generated 292 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:33.715197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.721027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.721316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.721605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.751032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] MFA Rules not processed for user `b5843ea9864340cebf453f4b348811a0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:33.752284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.784938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b277751-f1ab-4d2f-9c1c-4f2ed7b3a76e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.793276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.794276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.794557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.794843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.795499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.797143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.812959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1051/2117] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:33.816585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.834501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a469af29-47b3-4e1c-9890-f2be6b6ab689 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.835590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1067/2118] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:33.843080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.843695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.843972 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.844265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.844913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.846546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.848196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.849762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.857346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1052/2119] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:33.861415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:33.871991 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea05a04b-0e68-4583-b23e-82331989ab5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.876626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.877264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.877542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.877812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] PATH_INFO: `/v3/projects/891c5612b5ab48ff884c3d33ae874249/users/6c877aaaf361487982fd62bf6bf22bf6/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.893989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] RBAC: Authorizing `identity:create_grant(project_id=891c5612b5ab48ff884c3d33ae874249, user_id=6c877aaaf361487982fd62bf6bf22bf6, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:33.896681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9fd91-f85d-42dc-aa79-5446e4ef6539 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:33.911139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe55c32-952f-4435-98d7-975f2faa9afa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:33.912469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1053/2120] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:33 2022] PUT /identity/v3/projects/891c5612b5ab48ff884c3d33ae874249/users/6c877aaaf361487982fd62bf6bf22bf6/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:33.924187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.924474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.924855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:33.941609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:33.960176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] MFA Rules not processed for user `6c877aaaf361487982fd62bf6bf22bf6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:33.961413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:33.970133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:33.970894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:33.971174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:33.971451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.014620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:34.044231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1054/2121] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:33 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:34.050716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.073881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.075468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.085887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1068/2122] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:33 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:34.144435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e223ca70-e014-41c8-aed8-4f661d88b87a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.149133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.149742 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.150019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.150325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.151098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.153023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.187490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.194406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1055/2123] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:34.197955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-754b1e34-04e0-4912-aeff-be2d85061c58 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.198583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.270285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec53b019-392d-48f8-9bce-bb69e85fe3c7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.272077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.272823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.273159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.273640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.275785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:34.276825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.277476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.277777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.278055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.278828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.280862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.309872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1056/2124] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:34 2022] POST /identity/v3/users => generated 294 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:34.311307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.321677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e00755f-bc5c-4452-935f-53e2420d7c70 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.326786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.327335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.327599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.327877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.328516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.330113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.341301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1057/2125] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:34.344417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.345241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.345834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.354942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1069/2126] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:34 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 168 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:34.355158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d82f7b8-b778-416d-b870-02888330e791 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.360072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.360470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.360781 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.361008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.361647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.363257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c9708d3-3fcf-44a3-8a46-3368fe5adcfc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.374658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1058/2127] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:34.378989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.389960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f680a6-7def-4dc8-9022-e4258ec57d46 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.394838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.395482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.395756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.396024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] PATH_INFO: `/v3/projects/610db391ac5041f4aacc3de933b68c0c/users/52b97b0fff8b4f349e766c462c116d88/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.412865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] RBAC: Authorizing `identity:create_grant(project_id=610db391ac5041f4aacc3de933b68c0c, user_id=52b97b0fff8b4f349e766c462c116d88, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.415603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.433692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1070/2128] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:34 2022] PUT /identity/v3/projects/610db391ac5041f4aacc3de933b68c0c/users/52b97b0fff8b4f349e766c462c116d88/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:34.444381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.444721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.445062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.484900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] MFA Rules not processed for user `52b97b0fff8b4f349e766c462c116d88`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:34.486280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:34.522294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:34.585301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1059/2129] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:34 2022] POST /identity/v3/auth/tokens => generated 3264 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:34.589633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.599732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:34.663960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26670d7f-7934-4d19-bc0c-be7ed3ed6132 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.669477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.670087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.670400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.670675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.671333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.673249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.706963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1071/2130] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:34 2022] POST /identity/v3/projects => generated 387 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:34.708208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.720415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-714d2735-3091-473e-a2f5-cb582ed5a271 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.781479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.782097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.782415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.782830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.783326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.785395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.810974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1060/2131] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:34 2022] POST /identity/v3/users => generated 300 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:34.812346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.824729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f11f0e-35f2-4d04-9cb2-89d4676a7dd0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.830314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.831051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.831173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.831479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.832174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.833755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.846686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1072/2132] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:34.851231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.861870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b59a1da-d3e3-4cb4-bf82-0c84fff21879 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.866477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.867126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.867401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.867669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] PATH_INFO: `/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7/users/51970e577f314826a1deff5dc313d3d3/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.882369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d687d9a6e4344dbcb650eb5e7e6c5de7, user_id=51970e577f314826a1deff5dc313d3d3, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.885924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.901023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1061/2133] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:34 2022] PUT /identity/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7/users/51970e577f314826a1deff5dc313d3d3/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:34.905445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:34.917578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac3c454-45ea-4789-a68b-c50790f5799b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:34.980471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:34.981075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:34.981357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:34.981629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:34.982291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:34.983872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96cfa65d-c62f-4eeb-b58e-2a1b3cdb1d2b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:34.994693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1073/2134] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:34.998835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:35.009691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-015e90d0-8837-4d0b-8b9f-107896f01fe2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:35.014480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:35.015119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:35.015397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:35.015668 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] PATH_INFO: `/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7/users/51970e577f314826a1deff5dc313d3d3/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:35.017807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=d687d9a6e4344dbcb650eb5e7e6c5de7, user_id=51970e577f314826a1deff5dc313d3d3, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:35.020578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:35.031522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1062/2135] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:34 2022] PUT /identity/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7/users/51970e577f314826a1deff5dc313d3d3/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:35.042640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:35.042931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:35.043234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:35.070530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] MFA Rules not processed for user `51970e577f314826a1deff5dc313d3d3`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:35.073072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:35.148864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1074/2136] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:35 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:35.160200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:35.217319 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b216a4ee-9d62-4f31-ba5d-b5df9df2c3ca admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:35.222773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac379-867a-48aa-b194-39354059469d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:35.223401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac379-867a-48aa-b194-39354059469d service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:35.223654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac379-867a-48aa-b194-39354059469d service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:35.223915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935ac379-867a-48aa-b194-39354059469d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:35.234065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ac379-867a-48aa-b194-39354059469d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:35.235803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935ac379-867a-48aa-b194-39354059469d service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:35.246380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1063/2137] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:35 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:35.865225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:35.868584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:35.877323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:35.884758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2acf4b97-b92d-41eb-8719-1a8a53dbae60 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:35.890092 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:35.890800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:35.891080 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:35.891349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:35.971200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:35.973989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d208b855-ceae-45d5-afe5-0f0ba7b163df service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:35.991495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1075/2138] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:35 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:36.809806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac379-867a-48aa-b194-39354059469d service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:36.811331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-935ac379-867a-48aa-b194-39354059469d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:36.832039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:36.877342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935ac379-867a-48aa-b194-39354059469d service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:36.881780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:36.882427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:36.882706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:36.882975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] PATH_INFO: `/v3/users/78c2f7d45fda41569d376f98316613f5` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:36.884874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] RBAC: Authorizing `identity:delete_user(user_id=78c2f7d45fda41569d376f98316613f5)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:36.886816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:36.934071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78c2f7d45fda41569d376f98316613f5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:36.940851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78c2f7d45fda41569d376f98316613f5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:36.948620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78c2f7d45fda41569d376f98316613f5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:36.949390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78c2f7d45fda41569d376f98316613f5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:36.963983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78c2f7d45fda41569d376f98316613f5'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:36.969886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1064/2139] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:36 2022] DELETE /identity/v3/users/78c2f7d45fda41569d376f98316613f5 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:36.974571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:36.975035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:36.975147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.001676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:37.003076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.065361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1076/2140] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:36 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:37.075560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.077002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.143917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2416f880-2ccd-49d7-94b3-9f0a13403a6b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:37.147577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:37.148190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.148466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.148737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.160967 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:37.162933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2952c592-46c0-48be-9aa9-86a0591b0a76 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:37.178476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1065/2141] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:37.260197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.261922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.275554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106ee75a-5a8e-4c5b-b6d4-0fe1f02e0f04 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:37.280342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:37.280977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.281259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.281536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] PATH_INFO: `/v3/users/b4726382ddb046b7965745189808bc43` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.282729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b4726382ddb046b7965745189808bc43)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:37.284674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:37.335053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b4726382ddb046b7965745189808bc43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.352056 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b4726382ddb046b7965745189808bc43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.353123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b4726382ddb046b7965745189808bc43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.353755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b4726382ddb046b7965745189808bc43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.363137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b4726382ddb046b7965745189808bc43'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.376819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1077/2142] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:37 2022] DELETE /identity/v3/users/b4726382ddb046b7965745189808bc43 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:37.381281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.381563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.381831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.409470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:37.422752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.465829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:37.500068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1066/2143] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:37 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:37.510362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.511966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.519352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.521167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.531332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:37.547705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053b1c66-861d-4d66-b23f-1aa01ba5bd49 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:37.555139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:37.555979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.556460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.556716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] PATH_INFO: `/v3/projects/68e6f0d5b8ae45faaccd4d665d683bd4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.558012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] RBAC: Authorizing `identity:delete_project(project_id=68e6f0d5b8ae45faaccd4d665d683bd4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:37.560482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:37.590668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a5ea27-eef9-4f28-b632-7761ae154fc8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:37.595195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:37.595807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.596155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.596363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.609712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:37.611197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:37.623284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1078/2144] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:37 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:37.636390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68e6f0d5b8ae45faaccd4d665d683bd4', 'request_id': 'req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.645225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68e6f0d5b8ae45faaccd4d665d683bd4', 'request_id': 'req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.653707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68e6f0d5b8ae45faaccd4d665d683bd4', 'request_id': 'req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.664236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] The token cache is being invalidate because project 68e6f0d5b8ae45faaccd4d665d683bd4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:37.664526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.666029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.671383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1067/2145] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:37 2022] DELETE /identity/v3/projects/68e6f0d5b8ae45faaccd4d665d683bd4 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:37.675537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.686087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b810a204-ec4c-45ef-b8f2-03c3a570022f service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:37.762985 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:37.763628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:37.763901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:37.764173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] PATH_INFO: `/v3/users/f8ff1a22d96844aca6e914650e7ee396` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:37.765431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8ff1a22d96844aca6e914650e7ee396)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:37.767712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:37.835159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8ff1a22d96844aca6e914650e7ee396'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.865528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8ff1a22d96844aca6e914650e7ee396'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.867579 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8ff1a22d96844aca6e914650e7ee396'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.869932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8ff1a22d96844aca6e914650e7ee396'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.884412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8ff1a22d96844aca6e914650e7ee396'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:37.898739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1079/2146] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:37 2022] DELETE /identity/v3/users/f8ff1a22d96844aca6e914650e7ee396 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:37.956443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:37.958978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:37.988874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b617957b-52a9-4cb7-b27b-3b4d6a7a037e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.095750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.096820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.097331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.097814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] PATH_INFO: `/v3/projects/a6709c79943049a18d7190f0a98b8d7b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.099796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6709c79943049a18d7190f0a98b8d7b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.103407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.158052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6709c79943049a18d7190f0a98b8d7b', 'request_id': 'req-a01691a3-6222-4b36-8a2b-60c8c6f45c06'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.166700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6709c79943049a18d7190f0a98b8d7b', 'request_id': 'req-a01691a3-6222-4b36-8a2b-60c8c6f45c06'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.173076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6709c79943049a18d7190f0a98b8d7b', 'request_id': 'req-a01691a3-6222-4b36-8a2b-60c8c6f45c06'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.177509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:38.187246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] The token cache is being invalidate because project a6709c79943049a18d7190f0a98b8d7b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:38.187541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.188568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.191036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1068/2147] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:37 2022] DELETE /identity/v3/projects/a6709c79943049a18d7190f0a98b8d7b => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:38.195004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.197401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:38.215226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c28676-a626-4ad5-a36c-879752b81470 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.262906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:38.295854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.297607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:38.328132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.328903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.329180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.329477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] PATH_INFO: `/v3/users/b5843ea9864340cebf453f4b348811a0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.331878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5843ea9864340cebf453f4b348811a0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.335827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.393254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5843ea9864340cebf453f4b348811a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.393760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a01691a3-6222-4b36-8a2b-60c8c6f45c06 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.399800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.400747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.401032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.401311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] PATH_INFO: `/v3/projects/66cd20831575476fb271efef898cc23f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.402503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=66cd20831575476fb271efef898cc23f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.404423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.407919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5843ea9864340cebf453f4b348811a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.408814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5843ea9864340cebf453f4b348811a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.410067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5843ea9864340cebf453f4b348811a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.420121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f7b74261-63bc-422a-ac05-0c556ee4977e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5843ea9864340cebf453f4b348811a0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.439768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1080/2148] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:38 2022] DELETE /identity/v3/users/b5843ea9864340cebf453f4b348811a0 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:38.463534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '66cd20831575476fb271efef898cc23f', 'request_id': 'req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.473697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '66cd20831575476fb271efef898cc23f', 'request_id': 'req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.482400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '66cd20831575476fb271efef898cc23f', 'request_id': 'req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.493977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] The token cache is being invalidate because project 66cd20831575476fb271efef898cc23f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:38.494275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.495120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d1e12f9c-3cd7-4588-a7ea-c7fe170729ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:38.496731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1069/2149] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:38 2022] DELETE /identity/v3/projects/66cd20831575476fb271efef898cc23f => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:38.503757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.504014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.504378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.528711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:38.529971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:38.597048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1081/2150] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:38.606376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.606684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.608427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.631381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:38.632651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:38.648615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1070/2151] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:38 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:38.655212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.664762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541eef6e-6487-4d26-81ef-04087438e177 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.670024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.670697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.670975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.671247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.672161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:38.672497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.674064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.693409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1082/2152] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:38.694465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.708756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0380e078-9237-43a2-8ad9-e6d975115aa0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.716733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.717667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.718008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.718381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.719054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.721050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.763195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1071/2153] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:38 2022] POST /identity/v3/projects => generated 377 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:38.767883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.779796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8210153b-5e94-4709-be45-906e48cf7b06 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.787071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.788952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:38.833668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:38.858130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.858835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.859118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.859390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.860598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.863156 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.888925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:38.896884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300f59c7-88bf-48be-a776-09696b733590 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.905823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.906588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1083/2154] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:38 2022] POST /identity/v3/users => generated 289 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:38.907001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.907373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.907791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] PATH_INFO: `/v3/projects/851c6adc6a38474daed4ba29127f618d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.909329 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] RBAC: Authorizing `identity:delete_project(project_id=851c6adc6a38474daed4ba29127f618d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.909828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.913251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.923415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5491f310-3b86-4603-8ecf-6b4db540c06f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.930234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.931400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.931702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.931984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.932703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:38.934403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:38.951559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1084/2155] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:38.955763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:38.980755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c146aca-a3c3-48e5-8eda-a18b25b6c5e8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:38.995151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:38.995854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:38.996250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:38.996724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:38.999005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.001652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.018016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '851c6adc6a38474daed4ba29127f618d', 'request_id': 'req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:39.026610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1085/2156] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:38 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:39.027963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.037855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '851c6adc6a38474daed4ba29127f618d', 'request_id': 'req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:39.051124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '851c6adc6a38474daed4ba29127f618d', 'request_id': 'req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:39.059032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413cd1fa-fbf1-47d8-b194-2e83446171a6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.072467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] The token cache is being invalidate because project 851c6adc6a38474daed4ba29127f618d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:39.072997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:39.074383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ad5dd2d0-693f-468b-a6cc-eadb2fcd0bdd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:39.082642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1072/2157] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:38 2022] DELETE /identity/v3/projects/851c6adc6a38474daed4ba29127f618d => generated 0 bytes in 292 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:39.085640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a363-b317-4e83-bf3f-12a8383eed04 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.086076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a363-b317-4e83-bf3f-12a8383eed04 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.086564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1193a363-b317-4e83-bf3f-12a8383eed04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.116404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1193a363-b317-4e83-bf3f-12a8383eed04 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:39.118848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1193a363-b317-4e83-bf3f-12a8383eed04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:39.141832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.142794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.143076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.143351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] PATH_INFO: `/v3/projects/492fad3e1408475f8d1f3c1724e4f88a/users/74c5046dfc37476194323d76bc477ed8/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.168957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=492fad3e1408475f8d1f3c1724e4f88a, user_id=74c5046dfc37476194323d76bc477ed8, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.171816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0c1720-94c7-4baf-a7ad-70e3ed75b1f7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.186543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1086/2158] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:39 2022] PUT /identity/v3/projects/492fad3e1408475f8d1f3c1724e4f88a/users/74c5046dfc37476194323d76bc477ed8/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:39.198113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.198797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.198797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.204068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1073/2159] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:39.212341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.212628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.212925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.232510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] MFA Rules not processed for user `74c5046dfc37476194323d76bc477ed8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:39.234001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:39.234118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:39.235681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:39.310292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1074/2160] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:39.313507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1087/2161] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:39.315030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.323749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.325539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abe9bb98-96cc-4545-9aff-ea4d700e15b9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.325692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:39.330559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.331226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.331567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.331839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.332493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:39.332790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.334537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.347870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1075/2162] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:39.352254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.353975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:39.363849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758a6d65-b123-4e7d-a61e-feda8101e74e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.369395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.370099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.370493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.370837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.371381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.373317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.385622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:39.405229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1076/2163] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:39.409741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.415764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c13b6f-efae-48df-ad22-0c3dc4efe792 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.470006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.470567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.470848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.471121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.489398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ef356-8130-42b1-9988-00ee0f7b8b38 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.494588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.495198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.495473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.495893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.496539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.498477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.525013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1077/2164] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/users => generated 293 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:39.535925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.537586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.539072 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.547560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c7e54b-19f8-40c6-ba0f-0384af402089 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.549111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1088/2165] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:39 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:39.552555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.553192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.553522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.553800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.554486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.556634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.570093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1078/2166] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:39.574374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.595510 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25f7f2b-6228-4b06-a4b6-ee7bb3bdd880 service glance] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.606797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.607425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.607706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.607978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.608813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.610465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4e5d2-c630-47e7-8640-60c03ffe0cf8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.622481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1089/2167] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:39 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:39.626517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.637664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28463545-7587-4fbd-a027-57b39c03dd1f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.643160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.643821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.644101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.644382 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] PATH_INFO: `/v3/projects/fcaacf59f34f4d1c89a7fea70b3cf742/users/0e13011cc13f4d89aeda97882afb83c0/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.663883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcaacf59f34f4d1c89a7fea70b3cf742, user_id=0e13011cc13f4d89aeda97882afb83c0, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.668529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.690789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1079/2168] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:39 2022] PUT /identity/v3/projects/fcaacf59f34f4d1c89a7fea70b3cf742/users/0e13011cc13f4d89aeda97882afb83c0/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:39.701183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.701472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.701751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.735069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] MFA Rules not processed for user `0e13011cc13f4d89aeda97882afb83c0`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:39.736332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:39.792914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:39.818218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1090/2169] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:39.822815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.901334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc37801-5217-4e1b-936e-888605ed952b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:39.906239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:39.906871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:39.907163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:39.907428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:39.908055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:39.909948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:39.938480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1080/2170] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/projects => generated 385 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:39.943584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:39.955228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a36d712-143f-4e3e-8ab2-78357badfdfd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.047983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.048712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.048975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.049275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.049916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.051876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.078451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1091/2171] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:39 2022] POST /identity/v3/users => generated 299 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:40.085531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.098021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d53112e-e74c-4178-af89-8d818f401e52 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.107260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.108361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.108754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.109127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.109895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.111691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.123610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1081/2172] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:40.127532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.139605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce723d7-f34a-4455-a227-242e7c915284 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.144667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.145322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.145622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.145892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] PATH_INFO: `/v3/projects/4b35cf3cc34243189cc019520764584f/users/f15adf40ac8b49b48f4525dfa7846602/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.161896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b35cf3cc34243189cc019520764584f, user_id=f15adf40ac8b49b48f4525dfa7846602, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.165464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.180018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1092/2173] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:40 2022] PUT /identity/v3/projects/4b35cf3cc34243189cc019520764584f/users/f15adf40ac8b49b48f4525dfa7846602/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:40.183973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.194699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f768506-4cb6-48c5-b265-fe8d18d9935b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.214776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:40.259957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.260562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.260839 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.261109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.261749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.263586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.275558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1082/2174] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:40 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:40.279694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.292558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1496fdeb-d36a-4690-98e0-d3ecf058a984 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.303393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.305334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.306194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.306944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.307246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.307522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] PATH_INFO: `/v3/projects/4b35cf3cc34243189cc019520764584f/users/f15adf40ac8b49b48f4525dfa7846602/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.309948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b35cf3cc34243189cc019520764584f, user_id=f15adf40ac8b49b48f4525dfa7846602, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.313198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6a54ba0-7ad9-4eb5-8ff7-3d09f7d125ee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.324877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1093/2175] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:40 2022] PUT /identity/v3/projects/4b35cf3cc34243189cc019520764584f/users/f15adf40ac8b49b48f4525dfa7846602/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:40.336536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.336840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.337121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.368986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] MFA Rules not processed for user `f15adf40ac8b49b48f4525dfa7846602`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:40.370216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.374638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d16c6c8-860a-4d92-a398-933b63951460 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.434040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.434680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.434953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.435221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.437053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.448498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1094/2176] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:40 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:40.461777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.463496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.476400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ecffc3-323a-4a0d-a580-27355504b92e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.482294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.482799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.482944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.483193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.494319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.495748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.505318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1095/2177] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:40 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:40.510620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.512074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df8b1d-21af-4ff8-bd57-abf0399ef032 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.525722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1083/2178] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:40 2022] GET /identity/v3/auth/tokens => generated 3264 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:40.540389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.559958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:40.610082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72c8494-caa5-4988-ba8a-cf33b46db6fc service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.617517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.617517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.617517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.617517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] PATH_INFO: `/v3/users/0e13011cc13f4d89aeda97882afb83c0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.618243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0e13011cc13f4d89aeda97882afb83c0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.619672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.685138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0e13011cc13f4d89aeda97882afb83c0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:40.699656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0e13011cc13f4d89aeda97882afb83c0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:40.700540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0e13011cc13f4d89aeda97882afb83c0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:40.701373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0e13011cc13f4d89aeda97882afb83c0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:40.711380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0e13011cc13f4d89aeda97882afb83c0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:40.726239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1096/2179] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:40 2022] DELETE /identity/v3/users/0e13011cc13f4d89aeda97882afb83c0 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:40.731366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.731586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.732064 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.754097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:40.755390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.778870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:40.839297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1084/2180] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:40 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:40.851271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:40.853467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:40.921994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b10c2b-bf4a-4ba9-a973-1d345c60fd50 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:40.927694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:40.928271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:40.928584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:40.928854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:40.941172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:40.942641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:40.954611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1097/2181] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:40 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:41.319866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:41.322716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:41.348851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f0d94b-fdd8-4f04-9b96-5ae63766ed0b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:41.358024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:41.358760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:41.359035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:41.359299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] PATH_INFO: `/v3/projects/fcaacf59f34f4d1c89a7fea70b3cf742` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:41.360410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcaacf59f34f4d1c89a7fea70b3cf742)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:41.362334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:41.411464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcaacf59f34f4d1c89a7fea70b3cf742', 'request_id': 'req-1e83d1ba-0f6b-4112-b8c2-d472159187f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.419228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcaacf59f34f4d1c89a7fea70b3cf742', 'request_id': 'req-1e83d1ba-0f6b-4112-b8c2-d472159187f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.428999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcaacf59f34f4d1c89a7fea70b3cf742', 'request_id': 'req-1e83d1ba-0f6b-4112-b8c2-d472159187f6'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.444713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] The token cache is being invalidate because project fcaacf59f34f4d1c89a7fea70b3cf742 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:41.445046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.445677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.448262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1085/2182] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:41 2022] DELETE /identity/v3/projects/fcaacf59f34f4d1c89a7fea70b3cf742 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:41.452340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:41.462069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb6da2-fef0-4444-be50-75f8732ecf45 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:41.499332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:41.525956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:41.526642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:41.526926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:41.527201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] PATH_INFO: `/v3/users/f15adf40ac8b49b48f4525dfa7846602` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:41.528338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f15adf40ac8b49b48f4525dfa7846602)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:41.530273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:41.577887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f15adf40ac8b49b48f4525dfa7846602'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.590878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f15adf40ac8b49b48f4525dfa7846602'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.591608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f15adf40ac8b49b48f4525dfa7846602'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.592259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f15adf40ac8b49b48f4525dfa7846602'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.600624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b1a07-f73c-4d23-9efd-b177bace04a9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f15adf40ac8b49b48f4525dfa7846602'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:41.615553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1098/2183] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:41 2022] DELETE /identity/v3/users/f15adf40ac8b49b48f4525dfa7846602 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:41.912627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:41.914438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:41.935560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e83d1ba-0f6b-4112-b8c2-d472159187f6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.000692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.001337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.001618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.001893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] PATH_INFO: `/v3/projects/4b35cf3cc34243189cc019520764584f` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.003070 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b35cf3cc34243189cc019520764584f)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.005067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.056281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b35cf3cc34243189cc019520764584f', 'request_id': 'req-63032a09-cf46-4792-a3d3-03bb349c4403'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.059093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:42.066030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b35cf3cc34243189cc019520764584f', 'request_id': 'req-63032a09-cf46-4792-a3d3-03bb349c4403'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.074345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b35cf3cc34243189cc019520764584f', 'request_id': 'req-63032a09-cf46-4792-a3d3-03bb349c4403'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.087431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] The token cache is being invalidate because project 4b35cf3cc34243189cc019520764584f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:42.087935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.088641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-63032a09-cf46-4792-a3d3-03bb349c4403 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.090236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1086/2184] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:41 2022] DELETE /identity/v3/projects/4b35cf3cc34243189cc019520764584f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:42.095150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.095673 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.095949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.120114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:42.120114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:42.200582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1099/2185] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:42.202212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.202582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.202859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.227002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:42.229181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:42.240343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1087/2186] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:42.246036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.256679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f261810-6055-49bc-91b4-b29a069f3830 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.262230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.262876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.263158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.263447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.264112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:42.264414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.266022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.286326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1100/2187] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:42.288756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.300929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07096949-9ff4-4c0c-b4b3-8ded766e655b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.306590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.307203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.307482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.307751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.308593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.310551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.340322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1088/2188] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/projects => generated 393 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:42.344564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.356154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aafb7973-a99c-4ddb-9ec2-67137ab146a1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.404514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:42.424564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.425310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.425664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.426224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.426965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.429250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.460699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1101/2189] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/users => generated 297 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:42.465623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.479203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93eb3576-b21b-4196-a5d0-8b81d9aa1aed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.484769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.485414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.485694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.485964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.486667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.488368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.499334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1089/2190] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:42.503941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.519740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c1c848f-19b8-41fa-b10a-38b7b3f36e7b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.525976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.526642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.527015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.527184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.527822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.529395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113cd1fd-9569-44bc-8618-070626cf163d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.542509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1102/2191] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:42.546882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.560723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee44ff2-2be3-4843-9236-61e4de0deee8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.565575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.566350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.566719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.567065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] PATH_INFO: `/v3/projects/05a62fa58499437faadd68ba165973bf/users/cb75e4145e184ce188c714687eb35891/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.590728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05a62fa58499437faadd68ba165973bf, user_id=cb75e4145e184ce188c714687eb35891, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.596266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.620532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1090/2192] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:42 2022] PUT /identity/v3/projects/05a62fa58499437faadd68ba165973bf/users/cb75e4145e184ce188c714687eb35891/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:42.632544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.633123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.634193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.704562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] MFA Rules not processed for user `cb75e4145e184ce188c714687eb35891`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:42.706346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:42.780069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1103/2193] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:42.784953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.853591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6144713e-b6e5-4552-b888-54ca8b7e1541 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.858314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.859066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.859342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.859627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.860255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.862118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.872226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.873605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:42.888030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444f563e-b184-4d4d-9616-be33ba8a165e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.894551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1091/2194] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/projects => generated 393 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:42.894691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.894691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.894691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.894691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] PATH_INFO: `/v3/users/6c877aaaf361487982fd62bf6bf22bf6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.895658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:42.895755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c877aaaf361487982fd62bf6bf22bf6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.897789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.951926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c877aaaf361487982fd62bf6bf22bf6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.965060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c877aaaf361487982fd62bf6bf22bf6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.965290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d301618-ffc4-4ed1-a072-8df9c28c6576 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:42.965753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c877aaaf361487982fd62bf6bf22bf6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.966501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c877aaaf361487982fd62bf6bf22bf6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.970238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:42.970875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.971153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.971548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:42.972144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:42.974215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:42.974905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e386243b-9f6b-464a-a6bf-36e735953e5f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c877aaaf361487982fd62bf6bf22bf6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:42.985509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=3,keystone:SELECT=93,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:42.991376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1104/2195] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:42 2022] DELETE /identity/v3/users/6c877aaaf361487982fd62bf6bf22bf6 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:42.997748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:42.998071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:42.998346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.003979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1092/2196] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/users => generated 303 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:43.004923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.006552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.020641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b2f548-dfef-4c58-9ed7-512278d940a5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.022359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:43.023578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.045906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:43.088184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.088914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.089256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.089589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.090400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.092521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.098933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1105/2197] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:42 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:43.106618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1093/2198] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:43.109816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.110509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.111261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.121473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e48379-35f6-4971-9f0c-3bf612d1e733 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.126146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.126830 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.127115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.127383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] PATH_INFO: `/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b/users/95accd47a53342698272470c6f3877e2/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.142569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb61ca9a2024000a2e56653c9f0a53b, user_id=95accd47a53342698272470c6f3877e2, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.145265 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.165712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1094/2199] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:43 2022] PUT /identity/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b/users/95accd47a53342698272470c6f3877e2/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:43.169828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.182735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d05b8c2-2ee2-44f9-9ec4-5188752ea530 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.245705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.246469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.246753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.247017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.256844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762e8a2-1591-420d-805d-6e66c1f209c7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.257028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.258243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.267893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.267893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.267893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.267893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.268251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.268251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b467def-ca19-4750-84ef-8ce21028bac9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.275063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1106/2200] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:43.277223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1095/2201] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:43 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:43.281399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.291574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b92ee-f31f-4ba3-982a-fe1e486e951c service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.296723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.297360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.297638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.297909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] PATH_INFO: `/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b/users/95accd47a53342698272470c6f3877e2/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.300413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb61ca9a2024000a2e56653c9f0a53b, user_id=95accd47a53342698272470c6f3877e2, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.303576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.318242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1107/2202] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:43 2022] PUT /identity/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b/users/95accd47a53342698272470c6f3877e2/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:43.336260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.337613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.337922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.367468 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] MFA Rules not processed for user `95accd47a53342698272470c6f3877e2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:43.368749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.418694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:43.445391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1096/2203] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:43 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:43.460392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.462795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.525017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11abf20f-bfa2-4f84-af80-db71d5b16c4b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.528972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.529595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.529882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.530144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.539813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.541420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.551083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1108/2204] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:43 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:43.572165 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.573580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.605376 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.607335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.620892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d140a1df-ee74-458f-8265-3aa9c66452dd service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.624669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.625268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.625551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.625827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.627679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:43.640218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:43.652161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f95788d-1867-498e-916f-8f6167c99892 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.657066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.657714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.657986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.658276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] PATH_INFO: `/v3/projects/891c5612b5ab48ff884c3d33ae874249` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.659397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] RBAC: Authorizing `identity:delete_project(project_id=891c5612b5ab48ff884c3d33ae874249)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.662291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.706891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.708521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.720543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1109/2205] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:43 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:43.721060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '891c5612b5ab48ff884c3d33ae874249', 'request_id': 'req-5e2e1bd2-330e-4351-af99-0aa78a9f9991'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.737211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '891c5612b5ab48ff884c3d33ae874249', 'request_id': 'req-5e2e1bd2-330e-4351-af99-0aa78a9f9991'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.745546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '891c5612b5ab48ff884c3d33ae874249', 'request_id': 'req-5e2e1bd2-330e-4351-af99-0aa78a9f9991'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.757817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] The token cache is being invalidate because project 891c5612b5ab48ff884c3d33ae874249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:43.758108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.760085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.762347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1097/2206] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:43 2022] DELETE /identity/v3/projects/891c5612b5ab48ff884c3d33ae874249 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:43.766461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:43.780455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2010a529-6617-4f90-876b-873f71d7ea00 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:43.845724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:43.847073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:43.847357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:43.847627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] PATH_INFO: `/v3/users/52b97b0fff8b4f349e766c462c116d88` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:43.848953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=52b97b0fff8b4f349e766c462c116d88)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:43.851305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:43.909433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '52b97b0fff8b4f349e766c462c116d88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.931955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '52b97b0fff8b4f349e766c462c116d88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.932921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '52b97b0fff8b4f349e766c462c116d88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.933609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '52b97b0fff8b4f349e766c462c116d88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.952398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '52b97b0fff8b4f349e766c462c116d88'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:43.973914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1110/2207] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:43 2022] DELETE /identity/v3/users/52b97b0fff8b4f349e766c462c116d88 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:44.405751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:44.407805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:44.425385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2e1bd2-330e-4351-af99-0aa78a9f9991 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:44.490766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:44.493551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:44.493980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:44.494311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] PATH_INFO: `/v3/projects/610db391ac5041f4aacc3de933b68c0c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:44.495717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] RBAC: Authorizing `identity:delete_project(project_id=610db391ac5041f4aacc3de933b68c0c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:44.497772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:44.517667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:44.560788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '610db391ac5041f4aacc3de933b68c0c', 'request_id': 'req-9aedc75b-702e-40de-9a25-49a16ecba30f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.570007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '610db391ac5041f4aacc3de933b68c0c', 'request_id': 'req-9aedc75b-702e-40de-9a25-49a16ecba30f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.577340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '610db391ac5041f4aacc3de933b68c0c', 'request_id': 'req-9aedc75b-702e-40de-9a25-49a16ecba30f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.588623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] The token cache is being invalidate because project 610db391ac5041f4aacc3de933b68c0c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:44.588907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.589572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.591354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1098/2208] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:44 2022] DELETE /identity/v3/projects/610db391ac5041f4aacc3de933b68c0c => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:44.595454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:44.597005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:44.613786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b28a25d-165b-472e-9828-7b7e007b49bf admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:44.653897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:44.686665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:44.687410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:44.687695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:44.687968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] PATH_INFO: `/v3/users/51970e577f314826a1deff5dc313d3d3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:44.689160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] RBAC: Authorizing `identity:delete_user(user_id=51970e577f314826a1deff5dc313d3d3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:44.691144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:44.747502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '51970e577f314826a1deff5dc313d3d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.760553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '51970e577f314826a1deff5dc313d3d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.761246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '51970e577f314826a1deff5dc313d3d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.761852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '51970e577f314826a1deff5dc313d3d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.770456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-72acbffd-b13a-4751-82f8-7adc23ebacbe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '51970e577f314826a1deff5dc313d3d3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:44.786850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1111/2209] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:44 2022] DELETE /identity/v3/users/51970e577f314826a1deff5dc313d3d3 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:45.082293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.083727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:45.097008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aedc75b-702e-40de-9a25-49a16ecba30f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.158368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.159007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.159277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.159542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] PATH_INFO: `/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.160672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d687d9a6e4344dbcb650eb5e7e6c5de7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.162667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.221375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd687d9a6e4344dbcb650eb5e7e6c5de7', 'request_id': 'req-1e3822d7-7391-4cb4-9e68-35acb4a18147'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:45.236509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd687d9a6e4344dbcb650eb5e7e6c5de7', 'request_id': 'req-1e3822d7-7391-4cb4-9e68-35acb4a18147'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:45.244930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd687d9a6e4344dbcb650eb5e7e6c5de7', 'request_id': 'req-1e3822d7-7391-4cb4-9e68-35acb4a18147'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:45.262696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] The token cache is being invalidate because project d687d9a6e4344dbcb650eb5e7e6c5de7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:45.263099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:45.264087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-1e3822d7-7391-4cb4-9e68-35acb4a18147 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:45.266192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1099/2210] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:45 2022] DELETE /identity/v3/projects/d687d9a6e4344dbcb650eb5e7e6c5de7 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:45.272803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.272803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.273018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.295595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:45.296828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:45.406798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1112/2211] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:45.416979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.417921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.418227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.440437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:45.441150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:45.450278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1100/2212] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:45.454584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.464583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf22cea-ffa6-4a8f-9de1-68efd1a5f304 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.469238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.469837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.470118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.470469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.471122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:45.471423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.472996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.485491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1113/2213] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:45.489772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.500314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbfa6910-e2e0-4180-8c7d-6113a7cf62b2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.505430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.506029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.506341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.506613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.507290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.509775 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.542534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1101/2214] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/projects => generated 367 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:45.543482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.556335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfd1d8c-fa46-4fbd-8821-bfe8fcc6cef0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.595254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:45.623451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.624135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.624387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.624660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.625432 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.627427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.660644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1114/2215] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/users => generated 284 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:45.662093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.672904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0092f602-d84b-4d04-bc51-dd970ff27c1a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.677645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.678286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.678571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.678841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.679543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.681916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.683423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:45.697585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1102/2216] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:45.699374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.710325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d73534-ad48-4a58-b0e7-809ec7355a14 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.715428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.716038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.716322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.716593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.717247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.718868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9242601-8805-47e3-9368-c5879b0e5e0f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.730157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1115/2217] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:45.734707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.746132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666a04b3-1356-480c-be14-f16d979cee84 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:45.750786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:45.751429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.751705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.751975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] PATH_INFO: `/v3/projects/b01ebc2a3d4240c9983cc5cabd917e44/users/88e6a9c63da14f6a8da2c18fac5e0046/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.768896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b01ebc2a3d4240c9983cc5cabd917e44, user_id=88e6a9c63da14f6a8da2c18fac5e0046, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:45.771644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:45.790414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1103/2218] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:45 2022] PUT /identity/v3/projects/b01ebc2a3d4240c9983cc5cabd917e44/users/88e6a9c63da14f6a8da2c18fac5e0046/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:45.799212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:45.799500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:45.799776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:45.832276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] MFA Rules not processed for user `88e6a9c63da14f6a8da2c18fac5e0046`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:45.833487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:45.909530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1116/2219] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:45.914040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:45.989659 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c561b1-ff45-4e46-9067-55c2d8487695 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.002771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.003344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.003625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.003893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.004536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.006470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.034674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1104/2220] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:45 2022] POST /identity/v3/projects => generated 367 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:46.038990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.050564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6653d7d-0d80-4601-a534-094ef7be0467 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.110234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.111015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.111291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.111558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.112214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.114125 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.138338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1117/2221] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:46 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:46.141450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.152786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8032aa68-7ab1-44ec-80cc-28d621e186c3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.157505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.158101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.158419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.158689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.159326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.160930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.171851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1105/2222] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:46.175956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.186008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da567c3-fd94-42f2-9c4f-3dcae6e39650 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.191030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.191693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.192018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.192246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] PATH_INFO: `/v3/projects/31ded47bc01142ce88f2592efd1b7258/users/afc5086e3ffe460ba89e89c977479aed/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.207528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ded47bc01142ce88f2592efd1b7258, user_id=afc5086e3ffe460ba89e89c977479aed, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.210167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.225431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1118/2223] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:46 2022] PUT /identity/v3/projects/31ded47bc01142ce88f2592efd1b7258/users/afc5086e3ffe460ba89e89c977479aed/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:46.229726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.240796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aad5e59-4d1d-43e3-bc53-2a2592e15a23 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.301049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.301649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.301926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.302196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.302862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.304798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-740cf4e0-e740-4c86-bedf-b3e54379d9a4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.314638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1106/2224] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:46 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:20:46.319124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.331964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c01246-6dce-4a2f-9c47-161a405b9db4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.337516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.338235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.338542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.338820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] PATH_INFO: `/v3/projects/31ded47bc01142ce88f2592efd1b7258/users/afc5086e3ffe460ba89e89c977479aed/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.340768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] RBAC: Authorizing `identity:create_grant(project_id=31ded47bc01142ce88f2592efd1b7258, user_id=afc5086e3ffe460ba89e89c977479aed, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.343846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.349350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:46.359975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1119/2225] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:46 2022] PUT /identity/v3/projects/31ded47bc01142ce88f2592efd1b7258/users/afc5086e3ffe460ba89e89c977479aed/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:46.371168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.371458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.371732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.405218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] MFA Rules not processed for user `afc5086e3ffe460ba89e89c977479aed`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:46.406459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:46.437870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:46.479893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1107/2226] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:46 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:46.491599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.494389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:46.559010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab8f5c02-dbb6-4bbf-97ac-e215d629e64d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.562721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.563320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.563594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.563856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.573476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.574925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.584682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1120/2227] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:46 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:46.667562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:46.677227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4fb7b8-71d0-4d0e-8502-fb41d6f36b8f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:46.681330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:46.681922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:46.682196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:46.682495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:46.741097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:46.742564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d340dcd-cbe8-4792-8b70-2952812fe981 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:46.754341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1108/2228] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:46 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:47.292020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:47.293471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:47.360802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13f81380-f862-4d14-b2d8-4396a7932661 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:47.365460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:47.366078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:47.366385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:47.366656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] PATH_INFO: `/v3/users/cb75e4145e184ce188c714687eb35891` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:47.368260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb75e4145e184ce188c714687eb35891)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:47.370160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:47.420530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb75e4145e184ce188c714687eb35891'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:47.433590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb75e4145e184ce188c714687eb35891'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:47.434319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb75e4145e184ce188c714687eb35891'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:47.434927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb75e4145e184ce188c714687eb35891'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:47.443624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb75e4145e184ce188c714687eb35891'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:47.457253 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1121/2229] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:47 2022] DELETE /identity/v3/users/cb75e4145e184ce188c714687eb35891 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:47.462137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:47.462641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:47.462865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:47.485558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:47.486844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:47.548873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1109/2230] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:47 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:47.559281 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:47.560724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:47.582489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:47.624161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcdbbc0f-d6b7-4b6d-b8c8-ebafb2c69d1d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:47.628671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:47.629384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:47.629764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:47.630178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:47.640466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:47.641891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:47.652292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1122/2231] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:47 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:48.032278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:48.033730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:48.047237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526cb2d4-24a3-49fd-a221-48904c0c00df None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:48.052356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:48.052999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.053286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.053566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] PATH_INFO: `/v3/projects/05a62fa58499437faadd68ba165973bf` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.054738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=05a62fa58499437faadd68ba165973bf)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:48.057357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:48.112759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05a62fa58499437faadd68ba165973bf', 'request_id': 'req-be6ecf4a-e7cf-4729-b549-dab09158d8bf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.122282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05a62fa58499437faadd68ba165973bf', 'request_id': 'req-be6ecf4a-e7cf-4729-b549-dab09158d8bf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.129074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05a62fa58499437faadd68ba165973bf', 'request_id': 'req-be6ecf4a-e7cf-4729-b549-dab09158d8bf'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.130492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:48.143970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] The token cache is being invalidate because project 05a62fa58499437faadd68ba165973bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:48.144260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.145056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.146792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1110/2232] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:48 2022] DELETE /identity/v3/projects/05a62fa58499437faadd68ba165973bf => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:48.151861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:48.161852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e267a6-c623-42d7-829b-2339c095e04e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:48.222242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:48.223142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.223423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.223693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] PATH_INFO: `/v3/users/95accd47a53342698272470c6f3877e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.225172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95accd47a53342698272470c6f3877e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:48.227812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:48.283262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95accd47a53342698272470c6f3877e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.296748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95accd47a53342698272470c6f3877e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.297466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95accd47a53342698272470c6f3877e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.298244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95accd47a53342698272470c6f3877e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.306762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4cac42f5-93e4-4071-9b2a-c0d8b3ea7877 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95accd47a53342698272470c6f3877e2'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.326169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1123/2233] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:48 2022] DELETE /identity/v3/users/95accd47a53342698272470c6f3877e2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:48.637122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:48.638636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:48.654232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be6ecf4a-e7cf-4729-b549-dab09158d8bf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:48.719622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:48.720386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.720673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.721033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] PATH_INFO: `/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.722312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb61ca9a2024000a2e56653c9f0a53b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:48.724428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:48.775209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb61ca9a2024000a2e56653c9f0a53b', 'request_id': 'req-41356075-9f80-4b4c-88f2-44be51d6d6dc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.783240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb61ca9a2024000a2e56653c9f0a53b', 'request_id': 'req-41356075-9f80-4b4c-88f2-44be51d6d6dc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.791465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb61ca9a2024000a2e56653c9f0a53b', 'request_id': 'req-41356075-9f80-4b4c-88f2-44be51d6d6dc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.802627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] The token cache is being invalidate because project ebb61ca9a2024000a2e56653c9f0a53b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:48.802913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.803571 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-41356075-9f80-4b4c-88f2-44be51d6d6dc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:48.805060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1111/2234] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:48 2022] DELETE /identity/v3/projects/ebb61ca9a2024000a2e56653c9f0a53b => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:48.810731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.811022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.811302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.831650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:48.832903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:48.878127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:48.901733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1124/2235] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:48.909805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.910086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.910389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.931445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:48.932832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:48.941153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1112/2236] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:48 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:48.945444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:48.956127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedd52e0-d7ea-47e9-9027-a6421630ed3e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:48.960879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:48.961536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.961797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.962068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.962759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:48.963065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:48.964650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:48.977084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1125/2237] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:48.981567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:48.990801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46198c79-bae3-4714-82d4-604a0a1f94ec None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:48.995969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:48.996568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:48.996844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:48.997112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:48.997753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:48.999678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.028400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1113/2238] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:48 2022] POST /identity/v3/projects => generated 359 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:49.032950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.044452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537ada2-81c3-4ba0-ac3e-a5a3e831220e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.101104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.101619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.101919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.102834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.103558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.105466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.129381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1126/2239] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:49 2022] POST /identity/v3/users => generated 280 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:49.134690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.145170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc3276c-2342-400e-b922-a0e9923c7c88 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.150466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.151056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.151332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.151597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.152231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.153819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.163868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1114/2240] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:49.168098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.179481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52a16ed-5ba0-4ae8-83bb-2746173eca09 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.184721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.185347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.185631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.185905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.186609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.188212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c646e665-3c57-4f38-b2cc-0b13410e3b6e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.201004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1127/2241] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:49 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:49.205320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.220249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e70d36e-bcf3-4420-9dbc-7874779096bb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.228392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.229634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.230292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.230579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] PATH_INFO: `/v3/projects/e5e5e01a270b455081b52448896682c4/users/9db2cc4b5f884cfc8f5c52d9c3386c52/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.250082 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e5e5e01a270b455081b52448896682c4, user_id=9db2cc4b5f884cfc8f5c52d9c3386c52, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.252901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.268083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1115/2242] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:49 2022] PUT /identity/v3/projects/e5e5e01a270b455081b52448896682c4/users/9db2cc4b5f884cfc8f5c52d9c3386c52/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:49.279941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.280228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.280499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.310074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] MFA Rules not processed for user `9db2cc4b5f884cfc8f5c52d9c3386c52`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:49.311429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:49.381935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1128/2243] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:49 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:49.416149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.418051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:49.437267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=9,keystone:SELECT=88,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:49.481014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5e4caf-8558-46a8-bce5-ce790e8cdd4f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.484727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.485335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.485692 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.485949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.499026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.500564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32033456-5dd8-4f48-ab8c-d6488c01998e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.511088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1116/2244] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:49.528211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.587439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:49.627706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f694f3d5-1a20-4bef-bf00-bbb2c52f13d7 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.636342 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.636902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.637184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.637455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] PATH_INFO: `/v3/users/9db2cc4b5f884cfc8f5c52d9c3386c52` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.638632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9db2cc4b5f884cfc8f5c52d9c3386c52)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.640529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.702523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9db2cc4b5f884cfc8f5c52d9c3386c52'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:49.723938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9db2cc4b5f884cfc8f5c52d9c3386c52'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:49.724835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9db2cc4b5f884cfc8f5c52d9c3386c52'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:49.725693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9db2cc4b5f884cfc8f5c52d9c3386c52'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:49.736371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9db2cc4b5f884cfc8f5c52d9c3386c52'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:49.751113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1129/2245] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:49 2022] DELETE /identity/v3/users/9db2cc4b5f884cfc8f5c52d9c3386c52 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:49.765589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.766102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.766194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.785553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:49.786823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:49.853764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1117/2246] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:49 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:49.866434 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:49.866990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:49.932972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d4805f-ecb6-4bc1-8a32-63e8f98ce233 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:49.936899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:49.937535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:49.937818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:49.938091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:49.952026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:49.953668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a685070b-2b80-451e-96d8-bd35e2bfed2d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:49.964413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1130/2247] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:49 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:50.347792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.349397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:50.364390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064b9b50-6cd1-4632-ab18-b61d1be6458b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.370554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.371401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.371841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.372114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] PATH_INFO: `/v3/projects/e5e5e01a270b455081b52448896682c4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.373296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] RBAC: Authorizing `identity:delete_project(project_id=e5e5e01a270b455081b52448896682c4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.375239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.428833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e5e5e01a270b455081b52448896682c4', 'request_id': 'req-0e86eb29-d2ef-4c41-95c8-df79389fc0df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:50.437924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e5e5e01a270b455081b52448896682c4', 'request_id': 'req-0e86eb29-d2ef-4c41-95c8-df79389fc0df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:50.444655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e5e5e01a270b455081b52448896682c4', 'request_id': 'req-0e86eb29-d2ef-4c41-95c8-df79389fc0df'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:50.456727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] The token cache is being invalidate because project e5e5e01a270b455081b52448896682c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:50.457013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:50.457663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0e86eb29-d2ef-4c41-95c8-df79389fc0df admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:50.459192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1118/2248] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:50 2022] DELETE /identity/v3/projects/e5e5e01a270b455081b52448896682c4 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:50.464780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.465235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.465369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.485942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:50.487163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:50.542859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:50.558426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1131/2249] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:50.560531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349704-a6d0-4572-8087-57200476783a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.560812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349704-a6d0-4572-8087-57200476783a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.561101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c349704-a6d0-4572-8087-57200476783a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.582419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5c349704-a6d0-4572-8087-57200476783a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:50.583702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5c349704-a6d0-4572-8087-57200476783a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:50.596619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1119/2250] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:50.597987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.613725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0dfb3-1669-4c9e-9a33-a43d1198f91d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.620433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.621313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.621758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.622221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.623341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:50.623797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.625902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.657684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1132/2251] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:50.662562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c349704-a6d0-4572-8087-57200476783a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.672922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c349704-a6d0-4572-8087-57200476783a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.681489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.682112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.682509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.682776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.683401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.685385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.725395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1120/2252] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:50 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:50.730210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.740717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3126483b-0ee0-40af-afc7-959a41ed3dc7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.805739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.806487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.806797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.807061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.807772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.809798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.836446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1133/2253] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:50 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:50.838676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.849646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24df7812-6505-405a-83e6-f0244cd94b72 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.854436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.855034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.855308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.855575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.856216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.857807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.868260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1121/2254] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:50.872419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.882607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741f9ca2-e943-4bdb-ba3a-c238c199c3ad admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.888894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.889762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.890226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.890826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.891858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.894211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9104e97e-dc92-4d5f-8b8a-2cfc617dc31a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.905859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1134/2255] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:50.910097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:50.915782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:50.922134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63761923-be1b-47d0-bce1-3fa63e152916 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:50.928135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:50.928811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.929219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.929547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] PATH_INFO: `/v3/projects/9be528f1f6724984ae8255726876e0cc/users/2808c5b8877c42ecbe93fe0324838c03/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:50.946728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9be528f1f6724984ae8255726876e0cc, user_id=2808c5b8877c42ecbe93fe0324838c03, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:50.949421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:50.962733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1122/2256] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:50 2022] PUT /identity/v3/projects/9be528f1f6724984ae8255726876e0cc/users/2808c5b8877c42ecbe93fe0324838c03/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:50.972891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:50.973176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:50.973446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.001544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] MFA Rules not processed for user `2808c5b8877c42ecbe93fe0324838c03`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:51.002786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:51.072704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1135/2257] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:50 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:51.077255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.143912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12d1a7-dfbc-4fd9-a3f0-c4612c4cf138 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.148617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.150115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.150444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.150719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.151363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.153267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.185364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1123/2258] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:51 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:51.189383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.200694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac7cd78-b9c5-4e08-a126-a02cf073b4c2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.253002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:51.262224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.262857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.263136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.263405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.264036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.265920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.288067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1136/2259] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:51 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:51.292900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.303113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee11fa-e2cd-4643-bda5-2ec50cb1d374 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.308883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.309651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.309959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.310229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.314691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.316397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.331449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1124/2260] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:51.332641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.343259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7dc6717-ea3e-42bc-9b31-263a0d0b809f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.348117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.348846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.349128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.349420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] PATH_INFO: `/v3/projects/dd04e354d3de4e72bebaf4a04b058625/users/b55b7c1e33b84f43b57958f5354714bc/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.366503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd04e354d3de4e72bebaf4a04b058625, user_id=b55b7c1e33b84f43b57958f5354714bc, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.369762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.384478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1137/2261] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:51 2022] PUT /identity/v3/projects/dd04e354d3de4e72bebaf4a04b058625/users/b55b7c1e33b84f43b57958f5354714bc/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:51.388546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.399126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce195aa-dc15-4bba-9a08-9b30bbf62f15 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.466455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.467324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.467324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.467626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.468236 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.469829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f426e6-b137-4c36-9f10-1db77b20bbf0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.480702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1125/2262] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:51 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:51.484752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.495277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a316681-1194-49ff-80b5-277f4f5719a5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.500108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.500780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.501062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.501344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] PATH_INFO: `/v3/projects/dd04e354d3de4e72bebaf4a04b058625/users/b55b7c1e33b84f43b57958f5354714bc/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.503299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd04e354d3de4e72bebaf4a04b058625, user_id=b55b7c1e33b84f43b57958f5354714bc, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.505914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.518320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1138/2263] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:51 2022] PUT /identity/v3/projects/dd04e354d3de4e72bebaf4a04b058625/users/b55b7c1e33b84f43b57958f5354714bc/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:51.529862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.530398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.530677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.563390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] MFA Rules not processed for user `b55b7c1e33b84f43b57958f5354714bc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:51.564471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:51.609654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:51.645956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.647688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:51.647941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1126/2264] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:51 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:51.659719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.661102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:51.737645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbb3f51d-3992-4f50-b510-74cf7f44e118 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.738580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4139f83a-f0fa-43cf-bdd6-c2d322b09eee admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:51.743085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.743860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.744110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.744408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.744603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:51.745227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.745506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.745783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] PATH_INFO: `/v3/users/88e6a9c63da14f6a8da2c18fac5e0046` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.746962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88e6a9c63da14f6a8da2c18fac5e0046)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.748916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.815729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88e6a9c63da14f6a8da2c18fac5e0046'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:51.831529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:51.833191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2eb507-5867-4856-a2bb-b9f06a36843c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:51.833548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88e6a9c63da14f6a8da2c18fac5e0046'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:51.834403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88e6a9c63da14f6a8da2c18fac5e0046'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:51.835142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88e6a9c63da14f6a8da2c18fac5e0046'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:51.845041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1127/2265] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:51.845561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88e6a9c63da14f6a8da2c18fac5e0046'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:51.864634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1139/2266] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:51 2022] DELETE /identity/v3/users/88e6a9c63da14f6a8da2c18fac5e0046 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:51.869724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:51.870038 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:51.870350 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:51.894529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:51.894529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:51.981083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1128/2267] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:51.994395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:51.995816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:52.042686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:INSERT=6,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:52.078552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a804c-8c1f-42a7-b1af-e7b6a39504e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:52.083440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:52.084178 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:52.084534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:52.084992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:52.097356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:52.099485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:52.111354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1140/2268] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:52.466954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:52.468578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:52.481305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ea4361-3229-4350-83f5-cd0b2887875f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:52.490089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:52.491035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:52.491538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:52.492014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] PATH_INFO: `/v3/projects/b01ebc2a3d4240c9983cc5cabd917e44` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:52.494670 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b01ebc2a3d4240c9983cc5cabd917e44)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:52.498844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:52.537012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:52.562857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b01ebc2a3d4240c9983cc5cabd917e44', 'request_id': 'req-b9a8820d-9530-4534-8a53-3019b126c23f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.570627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b01ebc2a3d4240c9983cc5cabd917e44', 'request_id': 'req-b9a8820d-9530-4534-8a53-3019b126c23f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.577271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b01ebc2a3d4240c9983cc5cabd917e44', 'request_id': 'req-b9a8820d-9530-4534-8a53-3019b126c23f'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.590019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] The token cache is being invalidate because project b01ebc2a3d4240c9983cc5cabd917e44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:52.590360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.591026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.592721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1129/2269] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:52 2022] DELETE /identity/v3/projects/b01ebc2a3d4240c9983cc5cabd917e44 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:52.596521 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:52.606364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410e17d4-be1e-4712-bf1a-42fa59bec9c4 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:52.672520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:52.673155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:52.673433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:52.673826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] PATH_INFO: `/v3/users/afc5086e3ffe460ba89e89c977479aed` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:52.674943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] RBAC: Authorizing `identity:delete_user(user_id=afc5086e3ffe460ba89e89c977479aed)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:52.676850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:52.728778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afc5086e3ffe460ba89e89c977479aed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.740681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afc5086e3ffe460ba89e89c977479aed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.741289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afc5086e3ffe460ba89e89c977479aed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.741896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afc5086e3ffe460ba89e89c977479aed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.749632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-84d99a59-5915-4156-a5de-178d2899fcdd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afc5086e3ffe460ba89e89c977479aed'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:52.762742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1141/2270] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:52 2022] DELETE /identity/v3/users/afc5086e3ffe460ba89e89c977479aed => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:53.056552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.058111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:53.072339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a8820d-9530-4534-8a53-3019b126c23f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.133678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.134338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.134619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.134891 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] PATH_INFO: `/v3/projects/31ded47bc01142ce88f2592efd1b7258` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.136012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31ded47bc01142ce88f2592efd1b7258)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.137942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.187754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31ded47bc01142ce88f2592efd1b7258', 'request_id': 'req-e90fa7b2-2954-4111-a3c3-f143cefd38f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:53.196823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31ded47bc01142ce88f2592efd1b7258', 'request_id': 'req-e90fa7b2-2954-4111-a3c3-f143cefd38f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:53.203095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31ded47bc01142ce88f2592efd1b7258', 'request_id': 'req-e90fa7b2-2954-4111-a3c3-f143cefd38f1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:53.214228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] The token cache is being invalidate because project 31ded47bc01142ce88f2592efd1b7258 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:53.214542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:53.215185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e90fa7b2-2954-4111-a3c3-f143cefd38f1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:53.216672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1130/2271] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:53 2022] DELETE /identity/v3/projects/31ded47bc01142ce88f2592efd1b7258 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:53.221533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359b43-6976-4440-824b-ca0753280632 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.221815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359b43-6976-4440-824b-ca0753280632 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.222089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e359b43-6976-4440-824b-ca0753280632 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.243387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4e359b43-6976-4440-824b-ca0753280632 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:53.244594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4e359b43-6976-4440-824b-ca0753280632 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:53.294028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:53.311616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1142/2272] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:53.319614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.319902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.320177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.343144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:53.344366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:53.352867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1131/2273] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:53.357324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e359b43-6976-4440-824b-ca0753280632 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.369482 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e359b43-6976-4440-824b-ca0753280632 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.375810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.375810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.375810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.375810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.376246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:53.376387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.378094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.392593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1143/2274] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:53.397077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.407471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88018a1e-15da-4cc3-b44c-7bb853b6b619 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.412397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.413004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.413285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.413582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.414216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.416921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.449947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1132/2275] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:53 2022] POST /identity/v3/projects => generated 375 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:53.454225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.466826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f03ea08-8c07-4d20-8667-56ed8d6e4317 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.534365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.535034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.535325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.535601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.536275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.538232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.565455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1144/2276] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:53 2022] POST /identity/v3/users => generated 288 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:53.570016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.581477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253cfcbd-840c-4ae2-b937-800a5be48f54 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.590330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.590330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.590330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.590330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.590768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.590987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.606969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1133/2277] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:53.611989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.623030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2414e4b9-5f54-42a6-83d3-209c02132b17 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.627963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.628754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.629040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.629386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.630020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.632246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2c8af3-dc89-4f13-a86f-8865ebfa3ab2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.645141 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1145/2278] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:53 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:53.650488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.655962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:53.663184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded0c8e5-afef-4d9f-ba50-663caca40b1c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.669985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.670724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.671283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.671841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] PATH_INFO: `/v3/projects/d1948c68a8384a0bbbf7f38885b5bce9/users/ecbf9db44a9548f5a0db618a466b5ea7/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.688528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d1948c68a8384a0bbbf7f38885b5bce9, user_id=ecbf9db44a9548f5a0db618a466b5ea7, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.691380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.707531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1134/2279] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:53 2022] PUT /identity/v3/projects/d1948c68a8384a0bbbf7f38885b5bce9/users/ecbf9db44a9548f5a0db618a466b5ea7/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:53.721893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.722546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.722847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.756603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] MFA Rules not processed for user `ecbf9db44a9548f5a0db618a466b5ea7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:53.757963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:53.839471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1146/2280] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:53.852643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:53.856595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:53.921628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569167fb-0b3e-4be0-8ed1-b779667f29fc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:53.925743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:53.926581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:53.926865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:53.927298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:53.939605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:53.941138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10c5003-0c92-4748-aa0b-72d68b5dc6f4 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:53.955871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1135/2281] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:53 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:54.528248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:54.529661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:54.577316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:54.598220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbd4975-51b7-460f-9081-683b001d08a6 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:54.604555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:54.605235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:54.605525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:54.605792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] PATH_INFO: `/v3/users/2808c5b8877c42ecbe93fe0324838c03` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:54.606945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2808c5b8877c42ecbe93fe0324838c03)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:54.608868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:54.673839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2808c5b8877c42ecbe93fe0324838c03'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:54.698662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2808c5b8877c42ecbe93fe0324838c03'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:54.699717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2808c5b8877c42ecbe93fe0324838c03'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:54.700437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2808c5b8877c42ecbe93fe0324838c03'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:54.709264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2808c5b8877c42ecbe93fe0324838c03'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:54.726054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1147/2282] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:54 2022] DELETE /identity/v3/users/2808c5b8877c42ecbe93fe0324838c03 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:54.732187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:54.732935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:54.740150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:54.763331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:54.768335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:54.843938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1136/2283] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:54.853830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:54.855291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:54.919918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52b8af5b-508f-4f0c-b864-1b034edc6b31 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:54.924158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:54.924781 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:54.925170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:54.925491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:54.936048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:54.937469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:54.948863 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1148/2284] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:54 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:55.246621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:55.248091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:55.261152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b2308f1-5a2d-4354-9006-a2382ac62de6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:55.266161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:55.266846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:55.267380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:55.267653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] PATH_INFO: `/v3/projects/9be528f1f6724984ae8255726876e0cc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:55.268816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9be528f1f6724984ae8255726876e0cc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:55.270831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:55.327033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9be528f1f6724984ae8255726876e0cc', 'request_id': 'req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.335734 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9be528f1f6724984ae8255726876e0cc', 'request_id': 'req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.343029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9be528f1f6724984ae8255726876e0cc', 'request_id': 'req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.355996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] The token cache is being invalidate because project 9be528f1f6724984ae8255726876e0cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:55.356225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.357030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.359289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1137/2285] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:55 2022] DELETE /identity/v3/projects/9be528f1f6724984ae8255726876e0cc => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:55.363962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:55.375774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64131e9-78b8-43ad-b799-27a3469b6f34 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:55.444022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:55.444651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:55.444928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:55.445204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] PATH_INFO: `/v3/users/b55b7c1e33b84f43b57958f5354714bc` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:55.446980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] RBAC: Authorizing `identity:delete_user(user_id=b55b7c1e33b84f43b57958f5354714bc)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:55.449157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:55.454646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:55.507563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b55b7c1e33b84f43b57958f5354714bc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.527920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b55b7c1e33b84f43b57958f5354714bc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.528830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b55b7c1e33b84f43b57958f5354714bc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.529644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b55b7c1e33b84f43b57958f5354714bc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.537277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b3d988c8-a088-468a-a10f-a5b25b91263f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b55b7c1e33b84f43b57958f5354714bc'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:55.551081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1149/2286] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:55 2022] DELETE /identity/v3/users/b55b7c1e33b84f43b57958f5354714bc => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:55.853812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:55.856019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:55.879697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:55.884278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebd6ad00-30b1-4bc2-baa8-faf53c33e065 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:55.972884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:55.973566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:55.973847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:55.974120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] PATH_INFO: `/v3/projects/dd04e354d3de4e72bebaf4a04b058625` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:55.975349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd04e354d3de4e72bebaf4a04b058625)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:55.977585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.035757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd04e354d3de4e72bebaf4a04b058625', 'request_id': 'req-7468d756-f289-4d6c-b78e-951eb8125456'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:56.044308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd04e354d3de4e72bebaf4a04b058625', 'request_id': 'req-7468d756-f289-4d6c-b78e-951eb8125456'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:56.052676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd04e354d3de4e72bebaf4a04b058625', 'request_id': 'req-7468d756-f289-4d6c-b78e-951eb8125456'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:56.065289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] The token cache is being invalidate because project dd04e354d3de4e72bebaf4a04b058625 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:56.065649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:56.066411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7468d756-f289-4d6c-b78e-951eb8125456 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:56.071749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1138/2287] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:55 2022] DELETE /identity/v3/projects/dd04e354d3de4e72bebaf4a04b058625 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:56.073017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541df-c9de-4124-a643-94319923eb68 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.073442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541df-c9de-4124-a643-94319923eb68 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.073631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57541df-c9de-4124-a643-94319923eb68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.097727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e57541df-c9de-4124-a643-94319923eb68 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:56.099001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e57541df-c9de-4124-a643-94319923eb68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:56.167215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1150/2288] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:56.174945 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.175229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.175500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.197585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:56.198918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:56.207433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1139/2289] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:56.212238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57541df-c9de-4124-a643-94319923eb68 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.223174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57541df-c9de-4124-a643-94319923eb68 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.229550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.230607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.230828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.231099 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.232104 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:56.232227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.234078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.251758 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1151/2290] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:56.253193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.266324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22832cb8-9a38-442c-ba7b-a3916b10560a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.272515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.273480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.273910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.274315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.275365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.277142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.309780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1140/2291] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:56 2022] POST /identity/v3/projects => generated 369 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:20:56.312739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.324627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3229f74-85e4-4723-9049-84d8f968bc84 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.390000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.390645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.390925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.391195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.392214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.394057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.418558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1152/2292] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:56 2022] POST /identity/v3/users => generated 285 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:56.420544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.431843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd8c3ae0-41e1-45dc-822f-b042376b591a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.436634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.437252 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.437532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.437802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.438475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.440067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.451448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1141/2293] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:56.455395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.460548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:56.466951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6622fec-ee1d-4263-859a-723701f59525 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.472840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.473477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.473760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.474037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.474709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.476288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39d0a78-25be-4737-bbc8-72736a599e63 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.487111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1153/2294] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:56.491291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.501813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c3c6b9-95e3-42b7-8fe0-9ec7161795c5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.506446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.507061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.507338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.507601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] PATH_INFO: `/v3/projects/182440aacb8e43f28b85a629c2e331af/users/b5d13e7e680d435cb2922d3a8b446019/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.523345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] RBAC: Authorizing `identity:create_grant(project_id=182440aacb8e43f28b85a629c2e331af, user_id=b5d13e7e680d435cb2922d3a8b446019, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.526094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.539256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1142/2295] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:56 2022] PUT /identity/v3/projects/182440aacb8e43f28b85a629c2e331af/users/b5d13e7e680d435cb2922d3a8b446019/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:56.550919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373917-6a22-4168-95c0-f491b5435cef None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.551305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373917-6a22-4168-95c0-f491b5435cef None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.551639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19373917-6a22-4168-95c0-f491b5435cef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.580108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-19373917-6a22-4168-95c0-f491b5435cef None None] MFA Rules not processed for user `b5d13e7e680d435cb2922d3a8b446019`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:56.581308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-19373917-6a22-4168-95c0-f491b5435cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:56.645007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1154/2296] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:56 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:56.657194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:56.658919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:56.698591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:56.725181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3d6e99-baa0-4b03-a18a-9881c44e8dce admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:56.729073 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:56.729671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:56.729948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:56.730218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:56.738669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:56.740098 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:56.748818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1143/2297] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:56 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:20:57.027526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19373917-6a22-4168-95c0-f491b5435cef None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.029453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-19373917-6a22-4168-95c0-f491b5435cef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.105577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19373917-6a22-4168-95c0-f491b5435cef None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.110194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.110849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.111126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.111398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] PATH_INFO: `/v3/users/09020c01783d4c9f9a5acb532a77693d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.112550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09020c01783d4c9f9a5acb532a77693d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.114470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:57.147656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.149976 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.166544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8c3531-a2f8-4354-bd71-5d3be8d51a4e service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.175352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.176621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.177077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.177826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] PATH_INFO: `/v3/users/ecbf9db44a9548f5a0db618a466b5ea7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.179686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ecbf9db44a9548f5a0db618a466b5ea7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.183141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:57.206195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09020c01783d4c9f9a5acb532a77693d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.221221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09020c01783d4c9f9a5acb532a77693d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.222046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09020c01783d4c9f9a5acb532a77693d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.222765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09020c01783d4c9f9a5acb532a77693d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.233471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-7f452d76-54b6-42de-a0ad-0eac148d0a83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09020c01783d4c9f9a5acb532a77693d'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.265976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:57.274079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1155/2298] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:57 2022] DELETE /identity/v3/users/09020c01783d4c9f9a5acb532a77693d => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:57.274954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ecbf9db44a9548f5a0db618a466b5ea7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.278176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.278808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.279420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.284248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ecbf9db44a9548f5a0db618a466b5ea7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.295848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ecbf9db44a9548f5a0db618a466b5ea7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.296577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ecbf9db44a9548f5a0db618a466b5ea7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.300712 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:57.302093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.312280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fcd64c4-9a5d-495d-874c-227c5124bc38 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ecbf9db44a9548f5a0db618a466b5ea7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:57.318798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1144/2299] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:20:57 2022] DELETE /identity/v3/users/ecbf9db44a9548f5a0db618a466b5ea7 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:20:57.323517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.323956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.324303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.344300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:57.345522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.365260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1156/2300] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:57.375962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.377647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.410743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1145/2301] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:57.421296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.422636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.440501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b0274b-3e2d-45c0-ba49-f46c6fc0c9b0 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.489238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:57.497501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.498123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.498447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.498718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.499055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9739374-1033-439a-89c7-081ff4b14841 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.504103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.505198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.505551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.506176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.510881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.512368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:57.519639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.521288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:57.522472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1157/2302] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:57.532385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1146/2303] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:20:57.898045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.900704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.913648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d7b6a4-f342-4630-97a8-74c9fad66784 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.926290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.927089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.927440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.927787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] PATH_INFO: `/v3/projects/d1948c68a8384a0bbbf7f38885b5bce9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.934875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d1948c68a8384a0bbbf7f38885b5bce9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.936871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:57.963220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:57.964978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:57.982665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aca349-2c6a-417d-89e1-847156c785fa service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:57.988449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:57.989397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:57.989397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:57.989975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] PATH_INFO: `/v3/projects/e0af279e53714aebbb1b15c9d7d48757` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:57.992079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0af279e53714aebbb1b15c9d7d48757)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:57.995369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.014352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd1948c68a8384a0bbbf7f38885b5bce9', 'request_id': 'req-5c6e2a09-c522-4ef9-942c-27c12174ad26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.020847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd1948c68a8384a0bbbf7f38885b5bce9', 'request_id': 'req-5c6e2a09-c522-4ef9-942c-27c12174ad26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.022589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:58.029939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd1948c68a8384a0bbbf7f38885b5bce9', 'request_id': 'req-5c6e2a09-c522-4ef9-942c-27c12174ad26'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.044187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] The token cache is being invalidate because project d1948c68a8384a0bbbf7f38885b5bce9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:58.044478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.045136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-5c6e2a09-c522-4ef9-942c-27c12174ad26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.046678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1158/2304] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:57 2022] DELETE /identity/v3/projects/d1948c68a8384a0bbbf7f38885b5bce9 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:58.049852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0af279e53714aebbb1b15c9d7d48757', 'request_id': 'req-19ebd449-c6f3-44bb-b826-d923de4f8c16'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.051625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.051911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.052186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.059121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0af279e53714aebbb1b15c9d7d48757', 'request_id': 'req-19ebd449-c6f3-44bb-b826-d923de4f8c16'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.065916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0af279e53714aebbb1b15c9d7d48757', 'request_id': 'req-19ebd449-c6f3-44bb-b826-d923de4f8c16'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.073217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:58.074505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:58.080386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] The token cache is being invalidate because project e0af279e53714aebbb1b15c9d7d48757 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:20:58.080683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.081338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19ebd449-c6f3-44bb-b826-d923de4f8c16 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:20:58.083184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1147/2305] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:20:57 2022] DELETE /identity/v3/projects/e0af279e53714aebbb1b15c9d7d48757 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:58.152721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1159/2306] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:20:58.161011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.161679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.161679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.184985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:58.184985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:58.193347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1148/2307] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:58.198775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.209425 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c6c2291-7719-4d6c-9e57-37d637b57801 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.214957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.215512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.215791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.216060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.216751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:20:58.217052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.218722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.232649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1160/2308] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:20:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:20:58.237697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.249015 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83503dd1-e32c-4a4c-ab33-e20a503675ec None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.257903 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.259494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.259904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.260175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.260860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.262888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.300034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1149/2309] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:20:58 2022] POST /identity/v3/projects => generated 385 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:58.304369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.314408 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1253e2f5-7684-4bcb-8422-14b5fc60ce22 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.371834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.372441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.372719 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.372989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.373641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.375646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.398441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1161/2310] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:20:58 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:20:58.403541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.414074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5705507-8947-4640-8195-b631803c1db9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.418980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.419568 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.419842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.420112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.420749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.422388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.435133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1150/2311] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:58.439399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.450261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150c3db1-ad4e-4359-8341-6713cc0d2913 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.454992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.455584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.455867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.456155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.456772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.458519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e16e42-f5db-4a84-9619-35ba9be1957d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.471241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1162/2312] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:20:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:20:58.475416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.486024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8648ee-2877-4f24-b1b2-3f82a3d53d56 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.490943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.491583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.491864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.492166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] PATH_INFO: `/v3/projects/ddae968572774fa5b3d0a561b5c1b085/users/cebdac97042f43ab9a900087d1ee6a3e/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.508441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddae968572774fa5b3d0a561b5c1b085, user_id=cebdac97042f43ab9a900087d1ee6a3e, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.511511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.525675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1151/2313] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:20:58 2022] PUT /identity/v3/projects/ddae968572774fa5b3d0a561b5c1b085/users/cebdac97042f43ab9a900087d1ee6a3e/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:20:58.530732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.531018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.531312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.550726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:20:58.551915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:58.595489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:20:58.611925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1163/2314] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:20:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:20:58.622053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:20:58.623761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:20:58.682170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d048f49-fd8b-4515-8a0c-02da2166a2de admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:20:58.688330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:20:58.688950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:20:58.689227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:20:58.689498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:20:58.700177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:20:58.701687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476affcd-400d-4a5e-b76f-ee69a12b894b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:20:58.711164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1152/2315] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:20:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:02.946870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:02.951062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:02.967068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fea9f4-e235-49c6-ba54-ff4e80667ceb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:02.973568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:02.974462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:02.975315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:02.975726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] PATH_INFO: `/v3/users/74c5046dfc37476194323d76bc477ed8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:02.977209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=74c5046dfc37476194323d76bc477ed8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:02.979713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:03.048757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '74c5046dfc37476194323d76bc477ed8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:03.068429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '74c5046dfc37476194323d76bc477ed8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:03.069315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '74c5046dfc37476194323d76bc477ed8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:03.071247 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '74c5046dfc37476194323d76bc477ed8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:03.087642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '74c5046dfc37476194323d76bc477ed8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:03.113723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1164/2316] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:02 2022] DELETE /identity/v3/users/74c5046dfc37476194323d76bc477ed8 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:03.115561 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec765ac6-25a4-4274-926a-f49d4d3eb7a7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:03.116056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec765ac6-25a4-4274-926a-f49d4d3eb7a7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:03.116226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec765ac6-25a4-4274-926a-f49d4d3eb7a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:03.146120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ec765ac6-25a4-4274-926a-f49d4d3eb7a7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:03.148065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ec765ac6-25a4-4274-926a-f49d4d3eb7a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:03.160925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:03.279500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:03.280989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:03.286386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1153/2317] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:03.461628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264f8801-cdfd-4fad-b963-f0c46a172ad0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:03.471140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:03.471514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:03.471796 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:03.472069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:03.489909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:03.491783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:03.502569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1165/2318] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:03 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:03.545696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:03.545979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:03.546295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:03.585088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] MFA Rules not processed for user `cebdac97042f43ab9a900087d1ee6a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:03.588230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:03.750991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:03.752921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:03.759889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1154/2319] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:03 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 210 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:03.769945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e4e619-61a3-4605-b6c2-98e0b2488a3b service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:03.775416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:03.776404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:03.776754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:03.777160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:03.777967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:03.780129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:03.822996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1166/2320] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:03 2022] POST /identity/v3/projects => generated 385 bytes in 69 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:03.824511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:03.838431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b003066-0f35-442f-81a7-bb8fe2edc29e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:03.925519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:03.927413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:03.941702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:03.943130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:03.943464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:03.943752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:03.944756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:03.948062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:03.989048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1155/2321] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:03 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:03.990103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.027285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:04.041825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f784ec4-d858-4342-ae57-c012b92f6d3c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.050365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.051114 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.051490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.051833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.052558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.057119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.070550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1156/2322] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:03 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:21:04.071481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.073825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6ea28b-59da-423c-8f89-9edb446ee272 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.081542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.082198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0496759c-89ca-4181-9c43-7c0c152379bb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.082406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.087060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.087478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] PATH_INFO: `/v3/projects/492fad3e1408475f8d1f3c1724e4f88a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.088764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=492fad3e1408475f8d1f3c1724e4f88a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.090731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.092170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.093134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.093791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.094123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] PATH_INFO: `/v3/projects/b90f352797cc49208763b206fd72546c/users/ce595054fcc44a4699160ec0d4fa5c99/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.133589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90f352797cc49208763b206fd72546c, user_id=ce595054fcc44a4699160ec0d4fa5c99, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.136407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.139817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:04.175604 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1157/2323] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:04 2022] PUT /identity/v3/projects/b90f352797cc49208763b206fd72546c/users/ce595054fcc44a4699160ec0d4fa5c99/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:04.179438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.203670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '492fad3e1408475f8d1f3c1724e4f88a', 'request_id': 'req-2d076790-97e3-4357-8c7d-1a562063a2c7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:04.216208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '492fad3e1408475f8d1f3c1724e4f88a', 'request_id': 'req-2d076790-97e3-4357-8c7d-1a562063a2c7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:04.229546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '492fad3e1408475f8d1f3c1724e4f88a', 'request_id': 'req-2d076790-97e3-4357-8c7d-1a562063a2c7'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:04.259323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] The token cache is being invalidate because project 492fad3e1408475f8d1f3c1724e4f88a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:04.259653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:04.260565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-2d076790-97e3-4357-8c7d-1a562063a2c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:04.266369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1167/2324] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:03 2022] DELETE /identity/v3/projects/492fad3e1408475f8d1f3c1724e4f88a => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:04.277914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57f95-8b0b-4749-b442-57c62bc532c9 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.278379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57f95-8b0b-4749-b442-57c62bc532c9 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.278891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de57f95-8b0b-4749-b442-57c62bc532c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.310824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1de57f95-8b0b-4749-b442-57c62bc532c9 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:04.312960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1de57f95-8b0b-4749-b442-57c62bc532c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:04.328163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1168/2325] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:21:04.328374 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e72337-aad6-4733-ae58-c9a4d04b95fa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.331624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.332024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.332302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.335285 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.336078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.336531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.337149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.338019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.339928 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.361002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:04.362336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:04.372564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1158/2326] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:21:04.377464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.379129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:04.399930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1169/2327] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:04.399930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.399930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73eab2da-4e97-493a-8af5-8f53c38956ac None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.402330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.402946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.403224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.403495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.404146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:21:04.404449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.406087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.419773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1170/2328] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:21:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:21:04.424004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.424627 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00c7e9d-cdc1-4a70-80f9-b7b98b16f7ed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.432283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.432968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.433258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.433527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] PATH_INFO: `/v3/projects/b90f352797cc49208763b206fd72546c/users/ce595054fcc44a4699160ec0d4fa5c99/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.439387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b90f352797cc49208763b206fd72546c, user_id=ce595054fcc44a4699160ec0d4fa5c99, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.440942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ae132a-2ce3-4d49-af81-d17c49c2bec5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.443194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.446954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.447984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.448357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.448701 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.449688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.451774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.462893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1159/2329] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:04 2022] PUT /identity/v3/projects/b90f352797cc49208763b206fd72546c/users/ce595054fcc44a4699160ec0d4fa5c99/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:21:04.496459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1171/2330] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:04 2022] POST /identity/v3/projects => generated 383 bytes in 73 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:21:04.497235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.595804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9afa59-5e23-41b2-bb08-e16074f091d9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.674740 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.675835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.676271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.676698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.677518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.680700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.725097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1160/2331] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:04 2022] POST /identity/v3/users => generated 292 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:21:04.730540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.753025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746e8995-6da9-468b-9834-1e54d3fd1140 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.766046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.767329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.767915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.768288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.769462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.771842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.788646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1172/2332] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:04.794033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.817777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2716c4cd-39d2-46b9-b655-1bd3620b9f6d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.839613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.841062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.841419 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.841761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.843162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.846123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50887ae0-40dd-4ff1-9786-0487d8c9358d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.866578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1161/2333] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:04 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:04.871494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:04.886760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9847ef2d-ef43-43f2-a4dd-4618eae42201 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:04.893054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:04.893830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.894177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.894556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] PATH_INFO: `/v3/projects/5c8fa4eb661c4e97bd14cebcd54b71e2/users/8b683f71cebf4df6be1572e872ad9f95/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.917848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c8fa4eb661c4e97bd14cebcd54b71e2, user_id=8b683f71cebf4df6be1572e872ad9f95, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:04.920684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:04.949458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1173/2334] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:04 2022] PUT /identity/v3/projects/5c8fa4eb661c4e97bd14cebcd54b71e2/users/8b683f71cebf4df6be1572e872ad9f95/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:04.960258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:04.961013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:04.962058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:04.972938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:05.006944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] MFA Rules not processed for user `8b683f71cebf4df6be1572e872ad9f95`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:05.009751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:05.111175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1162/2335] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:04 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:21:05.115727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.188257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:05.219513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba2946-b7d4-4ca3-8e67-d065f401b027 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.229687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.230600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.231071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.231497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.232323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.234598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.274597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1174/2336] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:05 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:05.278287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.316497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8da8a62-1b29-4a49-b058-c2728629d5c3 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.396939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.397702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.398032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.398400 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.399150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.401169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.428698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1163/2337] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:05 2022] POST /identity/v3/users => generated 291 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:05.435722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.448471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac65b657-6c5e-48df-89bf-69513fd0e3bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.455360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.456373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.456716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.457058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.458369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.460613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.479765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1175/2338] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:05.481719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.497650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c376ee36-5e16-4655-8662-963af7c91dc2 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.508268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.509097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.509376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.509645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.510337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.512158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea87e7e3-1183-44d4-a947-eb10b43a0818 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.532824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1164/2339] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:05 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:05.537452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.554601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd7c16b-4a01-4b8f-a8d8-dff294da74d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.563454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.574601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.574895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.575191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] PATH_INFO: `/v3/projects/d7c0a6f360e64abbaf9294e2efe97492/users/54945124b9774442b649d89a49d39fac/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.598577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7c0a6f360e64abbaf9294e2efe97492, user_id=54945124b9774442b649d89a49d39fac, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.601311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.618050 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1176/2340] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:05 2022] PUT /identity/v3/projects/d7c0a6f360e64abbaf9294e2efe97492/users/54945124b9774442b649d89a49d39fac/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:05.629935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.630227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.630536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.663338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] MFA Rules not processed for user `54945124b9774442b649d89a49d39fac`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:05.664784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:05.763332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1165/2341] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:05.768378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.864203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99287e29-7ab1-4a16-9834-5774941e7e4d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:05.874205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:05.875082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:05.875386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:05.875792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:05.876531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:05.879422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:05.920871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1177/2342] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:05 2022] POST /identity/v3/projects => generated 385 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:05.925139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:05.932971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:05.940173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a4fad2-0725-40df-8ca2-6181ad24fbad None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.034349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.035510 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.035956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.036453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.037360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.042867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.084956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1166/2343] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:05 2022] POST /identity/v3/users => generated 299 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:06.087834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:06.101691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e60da29-999a-409a-a93c-e875d0668aa0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.108235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.109447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.110062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.110628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.111441 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.114683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.143307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1178/2344] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:21:06.144640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:06.157386 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef07db-e2f3-4860-b6c5-bdc1efd0140c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.164608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.164608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.164608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.164608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] PATH_INFO: `/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5/users/7e641247ac764c9dbd9258067ac5c70c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.183245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f2f192175c4236bb48a0ad7c86b9b5, user_id=7e641247ac764c9dbd9258067ac5c70c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.185977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.210316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1167/2345] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:06 2022] PUT /identity/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5/users/7e641247ac764c9dbd9258067ac5c70c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:06.213216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:06.245955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404c66a0-4b57-4be0-b16f-7b6883df5166 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.366138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.368251 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.368643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.368995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.370755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.372824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47c70d2a-6732-4bfa-a6de-484baa7a0d13 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.392722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1179/2346] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:06 2022] GET /identity/v3/roles => generated 1420 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:06.393981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:06.409701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee2689a-f285-4636-996e-d050b1e06734 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.416616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.417519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.417949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.418426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] PATH_INFO: `/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5/users/7e641247ac764c9dbd9258067ac5c70c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.421121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50f2f192175c4236bb48a0ad7c86b9b5, user_id=7e641247ac764c9dbd9258067ac5c70c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.424408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.438564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1168/2347] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:06 2022] PUT /identity/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5/users/7e641247ac764c9dbd9258067ac5c70c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 45 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:06.518575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.518865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.519142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.556266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:06.571447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] MFA Rules not processed for user `7e641247ac764c9dbd9258067ac5c70c`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:06.572688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:06.689636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1180/2348] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:06 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:06.710183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:06.714743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:06.817293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30be2bf0-353c-425b-8229-b1f18dfa4172 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:06.822334 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:06.823207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:06.823643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:06.824841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:06.896993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:06.898599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:06.912501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1169/2349] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:06 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:08.862422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:08.875383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc954e06-3830-4dba-a4c1-c1dd9e49243d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:08.883615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:08.885880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:08.886472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:08.887079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:09.038484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:09.047911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bce7c0-b944-4226-b637-5845fa2a16d2 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:09.092308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1181/2350] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:08 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:10.090352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:10.100771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6210dcf2-4843-46e5-93f8-7e54e8096a0c service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:10.110062 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:10.110644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:10.111195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:10.111583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:10.136192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:10.140314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:10.144563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:10.166288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1170/2351] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:10 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 77 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:11.034060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905244-daaa-4189-9ffe-5825e2d70798 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:11.034592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905244-daaa-4189-9ffe-5825e2d70798 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:11.034692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a905244-daaa-4189-9ffe-5825e2d70798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:11.071807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7a905244-daaa-4189-9ffe-5825e2d70798 None None] MFA Rules not processed for user `ce595054fcc44a4699160ec0d4fa5c99`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:11.073428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7a905244-daaa-4189-9ffe-5825e2d70798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:11.195150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1182/2352] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:11 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:11.220752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:11.222481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:11.299087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a58472-3504-41d1-973b-0028bcc5bd28 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:11.303110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:11.303871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:11.304428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:11.305296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:11.314021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:11.465415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:11.468077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:11.501822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1171/2353] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:11 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:11.759256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f96ef-0e3c-46f5-bfcb-f94c3a8dbc9b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:11.765363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f96ef-0e3c-46f5-bfcb-f94c3a8dbc9b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:11.765870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d17f96ef-0e3c-46f5-bfcb-f94c3a8dbc9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:11.803130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d17f96ef-0e3c-46f5-bfcb-f94c3a8dbc9b None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:11.805866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d17f96ef-0e3c-46f5-bfcb-f94c3a8dbc9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:11.853306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:12.005411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1183/2354] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:21:11 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:12.066469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:12.068718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:12.224572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3c7fdf-29cd-4f94-8f0c-6b6bbefc20b8 service glance] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:12.236483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:12.238332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:12.238997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:12.239846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:12.250984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:12.253649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:12.263802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1172/2355] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:21:12 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:21:13.005067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15e45-bdce-4968-9cfa-9f8a83f54693 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:13.005553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15e45-bdce-4968-9cfa-9f8a83f54693 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:13.005649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b15e45-bdce-4968-9cfa-9f8a83f54693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:13.047493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a0b15e45-bdce-4968-9cfa-9f8a83f54693 None None] MFA Rules not processed for user `eb61948f41344ea481024a96cf03db37`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:13.049186 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a0b15e45-bdce-4968-9cfa-9f8a83f54693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:13.189665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1184/2356] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:13 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:14.875622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:14.954959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:14.997007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fdd6f-cf74-4f15-975b-63342f346e1d service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:15.003138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:15.004104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:15.004598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:15.005024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] PATH_INFO: `/v3/users/8b683f71cebf4df6be1572e872ad9f95` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:15.006689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b683f71cebf4df6be1572e872ad9f95)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:15.009404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:15.187496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b683f71cebf4df6be1572e872ad9f95'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:15.206623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b683f71cebf4df6be1572e872ad9f95'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:15.219647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b683f71cebf4df6be1572e872ad9f95'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:15.222281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b683f71cebf4df6be1572e872ad9f95'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:15.251457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b683f71cebf4df6be1572e872ad9f95'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:15.271620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1173/2357] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:14 2022] DELETE /identity/v3/users/8b683f71cebf4df6be1572e872ad9f95 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:15.278752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:15.279338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:15.279536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:15.323310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:15.325888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:15.519029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1185/2358] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:15 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:15.529139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:15.534682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:15.633992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f201cc2e-2c9b-481f-ab07-dd936bddf50e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:15.639490 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:15.640312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:15.640909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:15.641351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:15.655277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:15.657148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:15.672310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1174/2359] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:15 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:16.146576 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:16.148306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:16.172990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aade852c-7629-4c42-8d7d-5e494a4e2d32 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:16.188079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:16.189720 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:16.190490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:16.190599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] PATH_INFO: `/v3/projects/5c8fa4eb661c4e97bd14cebcd54b71e2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:16.193197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c8fa4eb661c4e97bd14cebcd54b71e2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:16.197338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:16.298704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c8fa4eb661c4e97bd14cebcd54b71e2', 'request_id': 'req-41af2e11-ef82-4887-821e-f52e6e211d31'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.309868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c8fa4eb661c4e97bd14cebcd54b71e2', 'request_id': 'req-41af2e11-ef82-4887-821e-f52e6e211d31'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.319564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c8fa4eb661c4e97bd14cebcd54b71e2', 'request_id': 'req-41af2e11-ef82-4887-821e-f52e6e211d31'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.346437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:16.369638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] The token cache is being invalidate because project 5c8fa4eb661c4e97bd14cebcd54b71e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:16.370541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.372279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.374392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1186/2360] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:16 2022] DELETE /identity/v3/projects/5c8fa4eb661c4e97bd14cebcd54b71e2 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:16.378079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:16.392421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e05f47c-c5c1-4507-adb2-b684f2091ca3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:16.480221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:16.480861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:16.481281 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:16.481735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] PATH_INFO: `/v3/users/54945124b9774442b649d89a49d39fac` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:16.483645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] RBAC: Authorizing `identity:delete_user(user_id=54945124b9774442b649d89a49d39fac)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:16.486449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:16.490547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:16.582913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '54945124b9774442b649d89a49d39fac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.594730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '54945124b9774442b649d89a49d39fac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.607752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '54945124b9774442b649d89a49d39fac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.608678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '54945124b9774442b649d89a49d39fac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.635728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '54945124b9774442b649d89a49d39fac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:16.647271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1175/2361] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:16 2022] DELETE /identity/v3/users/54945124b9774442b649d89a49d39fac => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:21:17.049764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:17.051593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:17.068714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41af2e11-ef82-4887-821e-f52e6e211d31 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:17.167461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:17.168303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:17.168725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:17.169004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] PATH_INFO: `/v3/projects/d7c0a6f360e64abbaf9294e2efe97492` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:17.170167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7c0a6f360e64abbaf9294e2efe97492)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:17.172232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:17.255654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7c0a6f360e64abbaf9294e2efe97492', 'request_id': 'req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.264428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7c0a6f360e64abbaf9294e2efe97492', 'request_id': 'req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.279483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7c0a6f360e64abbaf9294e2efe97492', 'request_id': 'req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.300797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] The token cache is being invalidate because project d7c0a6f360e64abbaf9294e2efe97492 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:17.301840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.303197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.309145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1187/2362] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:17 2022] DELETE /identity/v3/projects/d7c0a6f360e64abbaf9294e2efe97492 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:17.310155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:17.317727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:17.361940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8c0c739-9ac0-441f-8c01-debcead4ae7c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:17.470008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:17.470874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:17.471154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:17.471617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] PATH_INFO: `/v3/users/7e641247ac764c9dbd9258067ac5c70c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:17.472957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7e641247ac764c9dbd9258067ac5c70c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:17.474934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:17.556846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7e641247ac764c9dbd9258067ac5c70c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.574238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7e641247ac764c9dbd9258067ac5c70c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.590232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7e641247ac764c9dbd9258067ac5c70c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.591573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7e641247ac764c9dbd9258067ac5c70c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.621351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8119f23f-150a-4c87-8771-e16b890af028 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7e641247ac764c9dbd9258067ac5c70c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:17.625054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:17.634995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1176/2363] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:17 2022] DELETE /identity/v3/users/7e641247ac764c9dbd9258067ac5c70c => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:18.065739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.067478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:18.099862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81b3abf-1ccd-4e83-bc9e-30a2d615ba0f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.175202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.175847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.176126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.176397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] PATH_INFO: `/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.177543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50f2f192175c4236bb48a0ad7c86b9b5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.179522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.211979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:18.246514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50f2f192175c4236bb48a0ad7c86b9b5', 'request_id': 'req-f698134a-d5b2-413d-aed2-5bda7c0948c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:18.253450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50f2f192175c4236bb48a0ad7c86b9b5', 'request_id': 'req-f698134a-d5b2-413d-aed2-5bda7c0948c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:18.265193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50f2f192175c4236bb48a0ad7c86b9b5', 'request_id': 'req-f698134a-d5b2-413d-aed2-5bda7c0948c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:18.289585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] The token cache is being invalidate because project 50f2f192175c4236bb48a0ad7c86b9b5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:18.290449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:18.291440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-f698134a-d5b2-413d-aed2-5bda7c0948c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:18.293741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1188/2364] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:18 2022] DELETE /identity/v3/projects/50f2f192175c4236bb48a0ad7c86b9b5 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:18.298651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.298943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.299253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.323683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:18.325183 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:18.425195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1177/2365] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:18.433625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.434179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.434453 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.459794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:18.461675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:18.473558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1189/2366] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:18.478789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.495981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b4b9b5-2dc4-4913-8615-132e16d64b4e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.506989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.508985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.509403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.509779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.512096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:21:18.513276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.517675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.556045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1178/2367] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:21:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:21:18.561133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.577708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bb054b-1450-477a-9c07-4195ac5e21fc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.585541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.586871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.587187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.587525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.588431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.591020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.636445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1190/2368] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:18 2022] POST /identity/v3/projects => generated 371 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:21:18.640255 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.655110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c6f3735-3efb-4f7d-9174-2766a610ffdc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.753215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.753977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.754357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.754696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.755543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.757831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.796036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1179/2369] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:18 2022] POST /identity/v3/users => generated 286 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:18.796924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.815835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b92e26-b761-40ba-a89a-f08e4434e064 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.832688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.833313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.833588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.833865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.834541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.836993 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.858034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1191/2370] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:18.859606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.873342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948d2239-4949-4ccb-9d91-0f6817f47d0b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.880474 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.882279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.882764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.883269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.884108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.886167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fbb6381-e90a-4e2d-9abf-619667ea9ccf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.918326 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:18.918506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1180/2371] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:18 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:18.933895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1ca8e00-3942-4983-bd94-6824a6b60067 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:18.942160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:18.942992 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:18.943295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:18.943614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] PATH_INFO: `/v3/projects/7ca6dbede37a49528a555112bcfd816a/users/e2f26641d43843eebbeb4eca40f306d2/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:18.969825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ca6dbede37a49528a555112bcfd816a, user_id=e2f26641d43843eebbeb4eca40f306d2, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:18.973026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:18.998221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1192/2372] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:18 2022] PUT /identity/v3/projects/7ca6dbede37a49528a555112bcfd816a/users/e2f26641d43843eebbeb4eca40f306d2/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:21:19.004497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436afb-6cf7-4a15-aa2c-fb93002476e7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:19.004800 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436afb-6cf7-4a15-aa2c-fb93002476e7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:19.005088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18436afb-6cf7-4a15-aa2c-fb93002476e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:19.032152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-18436afb-6cf7-4a15-aa2c-fb93002476e7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:19.033689 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-18436afb-6cf7-4a15-aa2c-fb93002476e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:19.137080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1181/2373] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:19.147593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:19.150399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:19.275684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6702ea0a-ffbe-4dd0-9ebf-e85057b52f6d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:19.280168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:19.281022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:19.281508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:19.282217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:19.294812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:19.297895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:19.317763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1193/2374] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:19 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:21:20.445861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61d4-f346-4e53-8a91-ff8834d972ff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:20.445861 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61d4-f346-4e53-8a91-ff8834d972ff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:20.446795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ab61d4-f346-4e53-8a91-ff8834d972ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:20.480638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:20.540774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f9ab61d4-f346-4e53-8a91-ff8834d972ff None None] MFA Rules not processed for user `e2f26641d43843eebbeb4eca40f306d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:20.543391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f9ab61d4-f346-4e53-8a91-ff8834d972ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:20.734219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1182/2375] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:20 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:20.747459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:20.750726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:20.881779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1de072-f9f3-4c04-9656-6fb26f06a77e service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:20.887652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:20.888419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:20.888874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:20.889480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:20.903968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:20.910669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:20.916926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0dbbf5-a9dc-4dd7-b24c-4946c31a94b2 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:20.942028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1194/2376] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:20 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:22.856085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad10e7-ba29-45db-9997-df1003a9d06d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:22.856631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad10e7-ba29-45db-9997-df1003a9d06d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:22.856801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ad10e7-ba29-45db-9997-df1003a9d06d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:22.903597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-63ad10e7-ba29-45db-9997-df1003a9d06d None None] MFA Rules not processed for user `e2f26641d43843eebbeb4eca40f306d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:22.905721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-63ad10e7-ba29-45db-9997-df1003a9d06d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:22.933752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1183/2377] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:22 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:25.739390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:25.739953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:25.740058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:25.760101 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] MFA Rules not processed for user `cebdac97042f43ab9a900087d1ee6a3e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:25.761371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:25.818617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1195/2378] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:25 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:28.499401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:28.499935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:28.499935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:28.520940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:28.522295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:28.599737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1184/2379] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:21:28 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:28.618860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:28.621067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:28.763805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7a7119-d956-4f11-9f91-173f4e9573bb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:28.778472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:28.779258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:28.779721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:28.780139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:28.792533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:28.794589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca629c03-2ed0-49fe-9e94-9d70233a5d88 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:28.814659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1196/2380] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:21:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 186 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:21:32.442186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:32.444357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:32.478014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f9ba714-3c92-44ab-9a5f-2f1e38361ad5 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:32.483495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:32.483999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:32.484276 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:32.484594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] PATH_INFO: `/v3/users/b5d13e7e680d435cb2922d3a8b446019` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:32.485890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b5d13e7e680d435cb2922d3a8b446019)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:32.488792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:32.573297 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b5d13e7e680d435cb2922d3a8b446019'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:32.584825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b5d13e7e680d435cb2922d3a8b446019'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:32.595003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b5d13e7e680d435cb2922d3a8b446019'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:32.595900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b5d13e7e680d435cb2922d3a8b446019'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:32.610596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b5d13e7e680d435cb2922d3a8b446019'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:32.620466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1185/2381] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:32 2022] DELETE /identity/v3/users/b5d13e7e680d435cb2922d3a8b446019 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:32.625420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:32.625989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:32.626094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:32.651665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:32.653192 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:32.754849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1197/2382] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:32.766489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:32.769275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:32.793304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:32.915455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e68f8f6-0875-4bf4-b709-0fa03531cee0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:32.920853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:32.921744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:32.922179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:32.922649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:32.937120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:32.940717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af537cf3-4f15-41a3-ae83-a683d04c4007 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:32.970504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1186/2383] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 205 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:33.529098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:33.531079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:33.558614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b580b-11c9-4130-915e-2ef01a3aa19b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:33.575014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:33.575685 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:33.575982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:33.576220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] PATH_INFO: `/v3/projects/182440aacb8e43f28b85a629c2e331af` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:33.578977 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] RBAC: Authorizing `identity:delete_project(project_id=182440aacb8e43f28b85a629c2e331af)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:33.581044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:33.612875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:33.669951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '182440aacb8e43f28b85a629c2e331af', 'request_id': 'req-936549ee-cff3-4dd2-9476-0f556bd9763c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:33.683486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '182440aacb8e43f28b85a629c2e331af', 'request_id': 'req-936549ee-cff3-4dd2-9476-0f556bd9763c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:33.696175 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '182440aacb8e43f28b85a629c2e331af', 'request_id': 'req-936549ee-cff3-4dd2-9476-0f556bd9763c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:33.714873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] The token cache is being invalidate because project 182440aacb8e43f28b85a629c2e331af was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:33.715741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:33.722980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-936549ee-cff3-4dd2-9476-0f556bd9763c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:33.728370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1198/2384] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:33 2022] DELETE /identity/v3/projects/182440aacb8e43f28b85a629c2e331af => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:21:33.730387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:33.730896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:33.731068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:33.756244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:33.757687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:33.843447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1187/2385] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:21:33.851655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:33.852017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:33.852301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:33.875228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:33.877400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:33.889259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1199/2386] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:21:33.893874 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:33.903986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e6a42e-7c74-46b7-8b83-37ad4f0f2f85 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:33.908576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:33.909171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:33.909448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:33.909717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:33.910395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:21:33.910697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:33.912294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:33.923995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1188/2387] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:21:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:21:33.928722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:33.939519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf6c6f3-0222-4ed2-a3dd-d7bf1978714d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:33.944235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:33.944845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:33.945187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:33.946054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:33.947036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:33.949131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:33.977336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1200/2388] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:21:33 2022] POST /identity/v3/projects => generated 397 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:33.981688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:33.991825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2398131-2e03-4920-bb2e-5f7f8ece7f3c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:34.060408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:34.061159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.061523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.061770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.062508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:34.064810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:34.094890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1189/2389] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:21:33 2022] POST /identity/v3/users => generated 299 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:21:34.095190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:34.110182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32a5040-1bfc-4736-a399-e9a7b985703a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:34.136161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:34.136795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.137078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.137388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.138215 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:34.139931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:34.160096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1201/2390] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:34.161641 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:34.175173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c40457d-6950-412e-a7e1-800c6bdc0867 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:34.185736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:34.189700 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.207111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.207454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.216259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:34.217952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a21f2e-bc08-44f8-be88-4d1554503225 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:34.235637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1190/2391] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:21:34 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:21:34.239882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:34.255895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15fbf4b-6adc-4eb5-a279-90cba47ca1ba admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:34.276839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:34.277535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.277819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.278700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] PATH_INFO: `/v3/projects/d8d12e451f9a4dada00ccd552f65024a/users/6548ee528f0646d28c188777d93f9bf7/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.336924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8d12e451f9a4dada00ccd552f65024a, user_id=6548ee528f0646d28c188777d93f9bf7, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:34.346093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:34.403831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1202/2392] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:21:34 2022] PUT /identity/v3/projects/d8d12e451f9a4dada00ccd552f65024a/users/6548ee528f0646d28c188777d93f9bf7/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:34.405650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.410312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.410312 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.496299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] MFA Rules not processed for user `6548ee528f0646d28c188777d93f9bf7`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:34.497699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:34.503622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:34.661949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1191/2393] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:34 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:34.676683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:34.679964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:34.864612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5e66993-f985-4ccd-b077-c6b5b64d8fbc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:34.888529 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:34.889458 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:34.889996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:34.890443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:34.916902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:34.916902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0fae29-7653-437d-ade2-ec39eab45884 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:34.942642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1203/2394] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:34 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:43.572358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:43.650487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-831764f5-3679-4435-ad51-093ac9b40373 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:43.655534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:43.656169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:43.656441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:43.656705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] PATH_INFO: `/v3/users/6548ee528f0646d28c188777d93f9bf7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:43.657831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=6548ee528f0646d28c188777d93f9bf7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:43.659766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:43.709680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6548ee528f0646d28c188777d93f9bf7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:43.717567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6548ee528f0646d28c188777d93f9bf7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:43.724832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6548ee528f0646d28c188777d93f9bf7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:43.725447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6548ee528f0646d28c188777d93f9bf7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:43.738220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6548ee528f0646d28c188777d93f9bf7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:43.745855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1192/2395] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:21:43 2022] DELETE /identity/v3/users/6548ee528f0646d28c188777d93f9bf7 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:43.750214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:43.750773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:43.750873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:43.771143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:43.772363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:43.824791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:43.842019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1204/2396] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:43 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:43.853428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:43.854869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:43.884103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:43.919301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b039a9d2-9961-44e5-a5c2-3a29a2bb99aa admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:43.922896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:43.923512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:43.923786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:43.924059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:43.933783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:43.935270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072d4643-1dae-4ba5-834e-80d2f1393341 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:43.947905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1193/2397] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:21:43 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:21:44.431961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:44.433562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:44.472064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4a3a71-c74d-489f-9b0a-a4439ac8e2eb None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:44.484871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:44.485534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:44.485812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:44.486086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] PATH_INFO: `/v3/projects/d8d12e451f9a4dada00ccd552f65024a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:44.487395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8d12e451f9a4dada00ccd552f65024a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:44.489384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:44.547684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8d12e451f9a4dada00ccd552f65024a', 'request_id': 'req-dee7b659-b05f-4553-97dc-9d91d24f6a7a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:44.560790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8d12e451f9a4dada00ccd552f65024a', 'request_id': 'req-dee7b659-b05f-4553-97dc-9d91d24f6a7a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:44.573097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8d12e451f9a4dada00ccd552f65024a', 'request_id': 'req-dee7b659-b05f-4553-97dc-9d91d24f6a7a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:44.591621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] The token cache is being invalidate because project d8d12e451f9a4dada00ccd552f65024a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:21:44.592259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:44.593197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:21:44.595499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1205/2398] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:21:44 2022] DELETE /identity/v3/projects/d8d12e451f9a4dada00ccd552f65024a => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:21:47.489061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd54dd-59d1-4618-8fa8-0597ad92e814 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:47.489578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd54dd-59d1-4618-8fa8-0597ad92e814 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:47.489736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fd54dd-59d1-4618-8fa8-0597ad92e814 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:47.511550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35fd54dd-59d1-4618-8fa8-0597ad92e814 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:47.512885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35fd54dd-59d1-4618-8fa8-0597ad92e814 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:47.587785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1194/2399] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:21:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:21:47.602446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:21:47.604347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:47.684199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee7b659-b05f-4553-97dc-9d91d24f6a7a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:21:47.689679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:21:47.690241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:47.690554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:47.690825 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:47.701173 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:21:47.703219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:21:47.712770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1206/2400] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:21:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:21:57.579654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=40 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:57.709158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=63,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:21:58.173449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:21:58.173737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:21:58.174011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:21:58.194311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] MFA Rules not processed for user `e2f26641d43843eebbeb4eca40f306d2`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:21:58.195582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:21:58.251071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1195/2401] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:21:58 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:01.033347 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:01.034099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:01.034988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:01.036161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:01.134948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3037ca0a-53b8-48f1-afc0-c58a42e13b35 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:01.138398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d211b4a0-5fcc-4cb4-9b4a-3672e1060e4d service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:01.140387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:01.141203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:01.141491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:01.141870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:01.143928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:01.144075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:01.144537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:01.144816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:01.145210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:01.147042 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:01.157146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:01.158788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb85b3e6-a058-43ce-952e-e3b605eba060 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:01.163330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:01.166373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0e39c3-d8ba-4c21-bbb0-4f5c11365e58 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:01.169738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1196/2402] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 137 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:22:01.177139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1207/2403] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:01 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:22:02.196487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.197074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.197567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.198557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1197/2404] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:22:02.204557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700938-fb53-4845-af8e-a0419e5a44b8 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.205537 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700938-fb53-4845-af8e-a0419e5a44b8 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.205909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6700938-fb53-4845-af8e-a0419e5a44b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.228739 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d6700938-fb53-4845-af8e-a0419e5a44b8 None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:02.229976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d6700938-fb53-4845-af8e-a0419e5a44b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:02.313094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1208/2405] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:02.325222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:02.327006 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:02.384869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1188c207-1bcb-4a4d-a97c-435e9d5407f7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:02.388683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:02.389318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.389600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.389868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.400636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:02.402051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56201e33-1fbd-4022-b908-d10730eb09bc service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:02.412526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1198/2406] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:22:02.492237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.492522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.492795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.494305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1209/2407] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:22:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:22:02.499397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6579fdb-2021-48a2-9b41-37374ec94261 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.499602 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6579fdb-2021-48a2-9b41-37374ec94261 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.499822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6579fdb-2021-48a2-9b41-37374ec94261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.521505 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e6579fdb-2021-48a2-9b41-37374ec94261 None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:02.522681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e6579fdb-2021-48a2-9b41-37374ec94261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:02.559034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:02.579843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1199/2408] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:02.592297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:02.594046 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:02.609842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece4b1a-34f2-4083-816a-f87fd2beaf9b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:02.613759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:02.614421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.614708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.614978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.623496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:02.624922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063b07b9-7308-4cd2-aee7-a62e6397c129 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:02.634197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1210/2409] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:02 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:22:02.761475 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d8ec-a2cf-4f14-8f0a-0cc4959ab537 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.761758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d8ec-a2cf-4f14-8f0a-0cc4959ab537 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.762032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f7d8ec-a2cf-4f14-8f0a-0cc4959ab537 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.763427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1200/2410] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:22:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:22:02.768754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.769039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.769308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.789596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:02.790799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:02.800143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1211/2411] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:22:02 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:22:02.812025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d8ec-a2cf-4f14-8f0a-0cc4959ab537 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:02.827536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f7d8ec-a2cf-4f14-8f0a-0cc4959ab537 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:02.832551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:02.833221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:02.833553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:02.833974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:02.853221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:02.855013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6de4b738-ab47-46e1-9d4e-bade63956b78 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:02.880232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1201/2412] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:02 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:22:05.632319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:05.642339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e9d7-4a9f-4405-a5be-6d1b83c1899e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:05.646077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:05.646937 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:05.647216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:05.647485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:05.649329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:05.712298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:05.714593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:05.716355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:05.726055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1212/2413] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:22:05 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:22:08.025007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350457e-4cda-4557-a88c-54aadc00b156 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:08.025989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350457e-4cda-4557-a88c-54aadc00b156 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:08.026458 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d350457e-4cda-4557-a88c-54aadc00b156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:08.048467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d350457e-4cda-4557-a88c-54aadc00b156 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:08.049980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d350457e-4cda-4557-a88c-54aadc00b156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:08.058797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1202/2414] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:22:08 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:08.072498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:08.084036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7cead6-87c9-46b1-bc23-e3fb670b5f14 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:08.090163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:08.090936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:08.091337 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:08.091711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:08.102926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:08.104609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:08.122464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1213/2415] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:22:08 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:22:18.042982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=15 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:18.111526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=8 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:32.286902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04830-3821-432d-baeb-087ab6943de1 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:32.287471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04830-3821-432d-baeb-087ab6943de1 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:32.287471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a04830-3821-432d-baeb-087ab6943de1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:32.309198 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-16a04830-3821-432d-baeb-087ab6943de1 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:32.310525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16a04830-3821-432d-baeb-087ab6943de1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:32.318658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1203/2416] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:22:32 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:32.341438 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:32.352477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f010fc-398e-46f9-a96e-a133293012af service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:32.357707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:32.358343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:32.358627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:32.358897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:32.368082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:32.369648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:32.377783 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1214/2417] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:22:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:22:33.238807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66659f00-a093-4ab3-be77-7151e06b29c5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:33.239272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66659f00-a093-4ab3-be77-7151e06b29c5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:33.239763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66659f00-a093-4ab3-be77-7151e06b29c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:33.267449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-66659f00-a093-4ab3-be77-7151e06b29c5 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:33.268686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-66659f00-a093-4ab3-be77-7151e06b29c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:33.276864 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1204/2418] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:22:33 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:33.294511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:33.310036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43eed25a-f286-4644-ae35-8de9f24818a4 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:33.316428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:33.317419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:33.317899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:33.318394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:33.332024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:33.335040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:33.345152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1215/2419] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:22:33 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:22:43.261291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:43.341064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=12 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:22:49.537145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:49.537613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:49.537772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:49.557446 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:22:49.558676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:49.565674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1205/2420] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:22:49 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:22:49.579026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:49.590062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e3315eb-5d42-4d08-bc38-a21e32659f6b service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:49.595179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:49.595767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:49.596051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:49.596325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:49.605171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:49.606775 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:49.615475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1216/2421] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:22:49 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:22:51.560564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:51.562993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:22:51.634304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197f2266-e1dd-4afe-8451-6eb29fbe1fea None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:51.638943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:51.639606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:51.639882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:51.640154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] PATH_INFO: `/v3/users/e2f26641d43843eebbeb4eca40f306d2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:51.641299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] RBAC: Authorizing `identity:delete_user(user_id=e2f26641d43843eebbeb4eca40f306d2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:51.643858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:51.704104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e2f26641d43843eebbeb4eca40f306d2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:51.717524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e2f26641d43843eebbeb4eca40f306d2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:51.732339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e2f26641d43843eebbeb4eca40f306d2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:51.733173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e2f26641d43843eebbeb4eca40f306d2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:51.749739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-3d178afe-ea0e-4f3a-a8ac-36e6c44b955d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e2f26641d43843eebbeb4eca40f306d2'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:51.759047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1206/2422] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:22:51 2022] DELETE /identity/v3/users/e2f26641d43843eebbeb4eca40f306d2 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:22:52.037127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:22:52.049137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd3e0dc-ede6-4dc6-bf4f-0a45aca74ce9 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:22:52.113316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:22:52.113699 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:22:52.113999 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:22:52.114295 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] PATH_INFO: `/v3/projects/7ca6dbede37a49528a555112bcfd816a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:22:52.115477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ca6dbede37a49528a555112bcfd816a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:22:52.118153 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:22:52.170353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ca6dbede37a49528a555112bcfd816a', 'request_id': 'req-1f354221-8fd0-4a9a-8e94-95e13db07aea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:52.176496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ca6dbede37a49528a555112bcfd816a', 'request_id': 'req-1f354221-8fd0-4a9a-8e94-95e13db07aea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:52.187555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ca6dbede37a49528a555112bcfd816a', 'request_id': 'req-1f354221-8fd0-4a9a-8e94-95e13db07aea'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:52.199541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] The token cache is being invalidate because project 7ca6dbede37a49528a555112bcfd816a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:22:52.199826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:52.200479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-1f354221-8fd0-4a9a-8e94-95e13db07aea admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:22:52.201968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1217/2423] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:22:52 2022] DELETE /identity/v3/projects/7ca6dbede37a49528a555112bcfd816a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:23:00.073948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d219-0b2f-49f4-b6da-7c58ece86fc5 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:23:00.074472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d219-0b2f-49f4-b6da-7c58ece86fc5 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:23:00.074566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5920d219-0b2f-49f4-b6da-7c58ece86fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:23:00.093762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-5920d219-0b2f-49f4-b6da-7c58ece86fc5 None None] MFA Rules not processed for user `eb61948f41344ea481024a96cf03db37`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:23:00.095010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5920d219-0b2f-49f4-b6da-7c58ece86fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:00.151417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1207/2424] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:23:00 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:23:02.196103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=57,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:23:10.144395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=81,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:23:24.032271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:23:24.032807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:23:24.032906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:23:24.052206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] MFA Rules not processed for user `8c30ec9810204a4685590c42707e41f8`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:23:24.053428 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:24.109485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1218/2425] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:23:24 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:23:34.102708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:23:39.867701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:23:39.869230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:23:39.869773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:23:39.891947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] MFA Rules not processed for user `96d18f6efa1f4c91a6f213c095c507a3`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:23:39.893752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:39.959086 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1208/2426] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:23:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:23:48.860698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:23:48.862146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:48.918878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb8956d-8ddf-4fa7-b210-83c598f9ebfd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:23:48.922463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:23:48.923068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:23:48.923346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:23:48.923615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:23:48.925339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:48.997123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:23:48.998589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa66c0f1-2a37-47b9-8e4d-6157f42e88a4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:23:49.009677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1219/2427] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:23:48 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:23:49.952093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=25 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:23:51.532383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:23:51.534074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:51.549718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b52757-07c8-46ab-95fc-a36199e75b7e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:23:51.554227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:23:51.554873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:23:51.555153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:23:51.555425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:23:51.557242 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:23:51.572628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:23:51.574623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:23:51.587118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1209/2428] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:23:51 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:23:59.004714 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=54 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:01.435381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:01.435945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:01.435945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:01.456426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:01.457727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:01.515525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1220/2429] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:24:01 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:01.563763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:01.565200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:01.633771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3798d-2731-47c1-bd8a-6630f2c8a0b3 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:01.639199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:01.639792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:01.640066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:01.640332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:01.648789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:01.650378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:01.658749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1210/2430] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:24:01 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:24:03.592319 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:03.593951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:03.612822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f2dba2-d482-4fc7-b437-f149173dc855 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:03.619345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:03.620895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:03.622200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:03.622708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:03.625019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:03.641266 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:03.643426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:03.657627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1221/2431] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:03 2022] GET /identity/v3/auth/tokens => generated 3270 bytes in 66 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:07.485392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:07.496828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ca55f6c-845d-4d94-ba87-f37b197a25b1 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:07.501681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:07.502402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:07.502931 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:07.503306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:07.505282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:07.584057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:07.586228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:07.597773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1211/2432] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:07 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:08.803691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:08.805841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:08.879875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:08.881332 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:08.913706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fadbfe6-2b88-4248-980d-ffae6dd5d046 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:08.919816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:08.920746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:08.921181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:08.921615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] PATH_INFO: `/v3/users/cebdac97042f43ab9a900087d1ee6a3e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:08.922985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cebdac97042f43ab9a900087d1ee6a3e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:08.925122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:08.929812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba792b95-f070-4477-b7f2-2d505ddbf6f8 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:08.936543 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:08.937769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:08.938087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:08.938483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:08.940674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:08.958369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:08.962038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:08.975805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1222/2433] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:08 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:08.991504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cebdac97042f43ab9a900087d1ee6a3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.000973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cebdac97042f43ab9a900087d1ee6a3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.011434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cebdac97042f43ab9a900087d1ee6a3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.012743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cebdac97042f43ab9a900087d1ee6a3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.015131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:09.028488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cebdac97042f43ab9a900087d1ee6a3e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.034727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1212/2434] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:24:08 2022] DELETE /identity/v3/users/cebdac97042f43ab9a900087d1ee6a3e => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:24:09.217635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:09.238852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc4f92f-d9ff-4e04-af7c-1f735477f6cb service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:09.340206 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:09.340847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:09.341127 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:09.341403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] PATH_INFO: `/v3/projects/ddae968572774fa5b3d0a561b5c1b085` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:09.342603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddae968572774fa5b3d0a561b5c1b085)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:09.344491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:09.373688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:09.427975 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddae968572774fa5b3d0a561b5c1b085', 'request_id': 'req-36e44aa5-f09d-4782-85e2-aefd1395c739'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.437437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddae968572774fa5b3d0a561b5c1b085', 'request_id': 'req-36e44aa5-f09d-4782-85e2-aefd1395c739'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.447807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddae968572774fa5b3d0a561b5c1b085', 'request_id': 'req-36e44aa5-f09d-4782-85e2-aefd1395c739'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.464034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] The token cache is being invalidate because project ddae968572774fa5b3d0a561b5c1b085 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:24:09.464469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.465413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.472623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:09.474071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1223/2435] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:24:09 2022] DELETE /identity/v3/projects/ddae968572774fa5b3d0a561b5c1b085 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:09.474963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:09.492527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e78cbf67-f593-47db-a3ee-4435e369cef0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:09.569691 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:09.570455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:09.571443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:09.571794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] PATH_INFO: `/v3/users/ce595054fcc44a4699160ec0d4fa5c99` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:09.573033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce595054fcc44a4699160ec0d4fa5c99)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:09.575148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:09.647948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce595054fcc44a4699160ec0d4fa5c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.660587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce595054fcc44a4699160ec0d4fa5c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.671450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce595054fcc44a4699160ec0d4fa5c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.672185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce595054fcc44a4699160ec0d4fa5c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.687519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-8b2e3111-3394-486c-88f6-fe27e83985ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce595054fcc44a4699160ec0d4fa5c99'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:09.695137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1213/2436] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:24:09 2022] DELETE /identity/v3/users/ce595054fcc44a4699160ec0d4fa5c99 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:09.901898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:09.903884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:09.926223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e44aa5-f09d-4782-85e2-aefd1395c739 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:09.988619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:09.989321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:09.989562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:09.989831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] PATH_INFO: `/v3/projects/b90f352797cc49208763b206fd72546c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:09.991023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b90f352797cc49208763b206fd72546c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:09.992950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.042460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b90f352797cc49208763b206fd72546c', 'request_id': 'req-21022b2f-becd-42b5-a45b-316311770443'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:10.049578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b90f352797cc49208763b206fd72546c', 'request_id': 'req-21022b2f-becd-42b5-a45b-316311770443'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:10.057803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b90f352797cc49208763b206fd72546c', 'request_id': 'req-21022b2f-becd-42b5-a45b-316311770443'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:10.070678 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] The token cache is being invalidate because project b90f352797cc49208763b206fd72546c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:24:10.070902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:10.071562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-21022b2f-becd-42b5-a45b-316311770443 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:10.073469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1224/2437] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:24:09 2022] DELETE /identity/v3/projects/b90f352797cc49208763b206fd72546c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:10.078663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.079238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.079238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.100603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:10.101964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:10.180169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1214/2438] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:10.189901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.190577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.190941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.213389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:10.214843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:10.228700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1225/2439] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:10.232547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.237751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:10.244586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04974fa5-de84-4a3a-bd09-6b5210ecfd2b None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.251260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.251978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.252264 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.252538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.253187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:24:10.253537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.255528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.271500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1215/2440] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:24:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:24:10.280305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.290661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b13957a-6a33-4530-92a0-c588fa55474e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.295606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.296222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.296477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.296744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.297374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.299293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.334297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1226/2441] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:24:10 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:10.337804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.350607 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3478e87-3fe9-449a-85e2-9d6d829ec560 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.414358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.415050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.415434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.415581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.416206 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.418820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.446290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1216/2442] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:24:10 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:10.450870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.461768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b7fd154-6ad6-4650-84be-b64ea3e08303 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.472133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.472761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.473040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.473307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.473954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.475567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.486203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1227/2443] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:24:10.490331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.503797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-913859e0-7e2e-4384-b544-c299a70a8c5a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.509213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.509915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.510304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.510654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.511370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.514180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2506ba07-6a16-461d-9a5f-3b7d9d93e19b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.527896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1217/2444] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:24:10.531724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.537191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:10.543335 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39868c80-9e06-4b83-b5fc-221f74d8a3bc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.554520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.555393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.555856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.556013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] PATH_INFO: `/v3/projects/b0f04feca7a349e5a656f166dae05354/users/7ba21f12574c4b9281d090651174788a/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.579431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f04feca7a349e5a656f166dae05354, user_id=7ba21f12574c4b9281d090651174788a, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.582057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.601479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1228/2445] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:24:10 2022] PUT /identity/v3/projects/b0f04feca7a349e5a656f166dae05354/users/7ba21f12574c4b9281d090651174788a/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:10.603286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.603764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.604079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.648970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:10.650403 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:10.739711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1218/2446] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:24:10.750867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:10.753214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:10.829077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351d9ba1-0422-4017-904d-af00b88560f1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:10.833740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:10.834580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:10.834986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:10.835454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:10.847063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:10.848306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:10.861130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1229/2447] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:10 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:11.136162 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:11.144160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:11.225941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:11.227631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:11.249922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523fc7fd-5efd-4068-a125-2fbb7359ed40 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:11.256398 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:11.257917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.259250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.260096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] PATH_INFO: `/v3/users/eb61948f41344ea481024a96cf03db37` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.262532 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb61948f41344ea481024a96cf03db37)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:11.265461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:11.287507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe2cf65-d4aa-43c6-ad6d-d3692be66af8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:11.309624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:11.311515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.312845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.313437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.315995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:11.341397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:11.344126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:11.349796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:11.361420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb61948f41344ea481024a96cf03db37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.364573 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1219/2448] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:11 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 229 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:11.380060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb61948f41344ea481024a96cf03db37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.380946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb61948f41344ea481024a96cf03db37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.381569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb61948f41344ea481024a96cf03db37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.392410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3a3bedc7-b7aa-4ce4-a127-04d098cba38a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb61948f41344ea481024a96cf03db37'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.413961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1230/2449] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:24:11 2022] DELETE /identity/v3/users/eb61948f41344ea481024a96cf03db37 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:11.611993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:11.624962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd388ef3-aeb3-4bd7-9245-9efb4f3b82f8 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:11.698187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:11.698852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.699158 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.699429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] PATH_INFO: `/v3/projects/a9a5c650ed23445bb585711f962d7e09` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.700567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a9a5c650ed23445bb585711f962d7e09)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:11.702649 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:11.758636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a9a5c650ed23445bb585711f962d7e09', 'request_id': 'req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.768909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a9a5c650ed23445bb585711f962d7e09', 'request_id': 'req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.776494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a9a5c650ed23445bb585711f962d7e09', 'request_id': 'req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.790156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] The token cache is being invalidate because project a9a5c650ed23445bb585711f962d7e09 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:24:11.790473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.791124 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6b8d3de1-2a98-4fbf-8a5b-9acdf46c8272 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:24:11.792688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1220/2450] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:24:11 2022] DELETE /identity/v3/projects/a9a5c650ed23445bb585711f962d7e09 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:11.798616 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.798901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.799172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.829965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:11.831277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:11.891006 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:11.911423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1231/2451] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:11.919300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.920157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.920157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.942962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:11.944213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:11.953335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1221/2452] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:11 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:11.957738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:11.969204 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f92915-ef72-4a64-992c-8d4edc1f1b8a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:11.979030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:11.979737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:11.980081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:11.981065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:11.981819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:24:11.982185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:11.983983 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:11.996344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1232/2453] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:24:12.000926 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.010143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b7251-1cd2-4634-bfc1-2d6836c8a324 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.016001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.016605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.016881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.017148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.018060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.020215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.055383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1222/2454] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:24:11 2022] POST /identity/v3/projects => generated 383 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:12.060942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.071275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fda8017c-2879-40d4-8e5a-63622e555ab6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.142155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.143708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.143708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.143708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.144333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.146503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.173419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1233/2455] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:24:12 2022] POST /identity/v3/users => generated 292 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:12.174662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.186461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3831b76-b61f-4e98-8eec-cf32de4403cb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.191449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.192582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.192790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.193166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.193914 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.195663 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.210369 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1223/2456] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:24:12.212194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.223230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a8818d-ecfc-4ce4-9f38-042d91bc6c1c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.229646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.230325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.230666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.231158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.231913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.233735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2f157e-b6b9-48bb-a321-c7c508437d05 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.245346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1234/2457] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:24:12.249657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.262676 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78d04321-71cf-484c-bb72-754100f8f90c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.268574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.269556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.269716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.270025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] PATH_INFO: `/v3/projects/9c9311c57fdd4250a16560bd8c2e0ffc/users/545b18260abe47ceb440d0713439c13b/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.287894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c9311c57fdd4250a16560bd8c2e0ffc, user_id=545b18260abe47ceb440d0713439c13b, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.290590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.315479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1224/2458] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:24:12 2022] PUT /identity/v3/projects/9c9311c57fdd4250a16560bd8c2e0ffc/users/545b18260abe47ceb440d0713439c13b/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:12.320645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.320933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.321209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.342914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:12.344174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:12.431472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1235/2459] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:12 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:12.439658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:12.441434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:12.474761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:12.513754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd70e75-72d1-46d0-9ec5-699e6ffbc56d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:12.520128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:12.521215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:12.521215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:12.521434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:12.532222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:12.534143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1cb72f-ed41-432a-9a42-c9a4812dd2e7 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:12.547083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1225/2460] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:15.476025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:15.478717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:15.682276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:15.730009 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef1bc9a-f930-4ceb-8110-8703898d2ab5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:15.737541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:15.738367 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:15.738647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:15.738919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:15.743162 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:15.836658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:15.837125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:15.837389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:15.884399 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] MFA Rules not processed for user `7ba21f12574c4b9281d090651174788a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:15.885612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:15.895106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:15.896897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:15.919260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1236/2461] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:15 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:15.986300 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1226/2462] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:15 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:24:15.990305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:15.992392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:16.009494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea90c4da-4b69-4c7d-bd5e-11254012d95b service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.015751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.016659 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.017292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.018847 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.019838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.022189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.084349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1237/2463] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:24:15 2022] POST /identity/v3/projects => generated 373 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:16.087141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:16.111683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-792957ab-fede-4f7c-b6b1-316b10c6b5ac None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.195630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.196434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.196815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.197187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.198085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.200802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.235584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1227/2464] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:24:16 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:24:16.236621 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:16.277958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cde9c25e-5e3f-4953-b5a5-e2c204cf4951 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.293772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.295626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.295912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.296182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.296840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.301331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.335917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1238/2465] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:24:16.341665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:16.357737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d0127c-2ab4-4255-a9d5-9ab2a3cf1a77 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.372718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.373506 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.373888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.374269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] PATH_INFO: `/v3/projects/9a7263aaa79b4900afd0bedd9969ece9/users/6cbc4765c42142a1aad99d6cc830e1b0/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.444131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7263aaa79b4900afd0bedd9969ece9, user_id=6cbc4765c42142a1aad99d6cc830e1b0, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.451096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.482043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1228/2466] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:24:16 2022] PUT /identity/v3/projects/9a7263aaa79b4900afd0bedd9969ece9/users/6cbc4765c42142a1aad99d6cc830e1b0/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:16.483752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:16.496140 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4966443-85c2-47e4-9769-428d08e2486c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.610580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.611444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.611727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.612001 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.613154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.614973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ac569-0b25-4656-bfcc-7cbe2809516f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.627383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1239/2467] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:24:16.631564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:16.656741 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02dd866-18d2-46cb-b819-644397bdf5cf admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:16.676479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:16.677672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:16.677672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:16.677672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] PATH_INFO: `/v3/projects/9a7263aaa79b4900afd0bedd9969ece9/users/6cbc4765c42142a1aad99d6cc830e1b0/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:16.683480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7263aaa79b4900afd0bedd9969ece9, user_id=6cbc4765c42142a1aad99d6cc830e1b0, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:16.686219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:16.707449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1229/2468] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:24:16 2022] PUT /identity/v3/projects/9a7263aaa79b4900afd0bedd9969ece9/users/6cbc4765c42142a1aad99d6cc830e1b0/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:24:18.991472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f2d1-0d6e-461a-86ea-127541536ba3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:18.992468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f2d1-0d6e-461a-86ea-127541536ba3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:18.992936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c0f2d1-0d6e-461a-86ea-127541536ba3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:19.082452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-47c0f2d1-0d6e-461a-86ea-127541536ba3 None None] MFA Rules not processed for user `545b18260abe47ceb440d0713439c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:19.083953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-47c0f2d1-0d6e-461a-86ea-127541536ba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:19.211944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:19.248741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1240/2469] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 259 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:19.267538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:19.270142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:19.293718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:19.466634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfd169a-f3c1-4b47-8d02-38e5139a365b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:19.483875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:19.484514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:19.488063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:19.488529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:19.519146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:19.521046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:19.541191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1230/2470] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:19 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:22.319905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:22.320404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:22.320516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:22.363961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] MFA Rules not processed for user `545b18260abe47ceb440d0713439c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:22.365664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:22.382717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1241/2471] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:22 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:22.930523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:23.019361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390432a3-0c27-4f07-862a-79874c358a14 service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:23.025808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:23.026538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:23.027809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:23.028149 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:23.031100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:23.048361 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:23.050150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b594082-4218-4cc1-b92e-a4f6bd57c60b service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:23.061389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1231/2472] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:22 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:23.721674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:23.723732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:23.845598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b528a86-3ca8-4548-ae49-51f717b021ae None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:23.850905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:23.852068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:23.852988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:23.853580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:23.857260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2f6e-ae4a-4edf-bdd9-87f4e3f51cbf None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:23.857780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2f6e-ae4a-4edf-bdd9-87f4e3f51cbf None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:23.858151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ae2f6e-ae4a-4edf-bdd9-87f4e3f51cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:23.858658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:23.873592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:23.875765 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:23.888475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1242/2473] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:23 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:23.907001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f6ae2f6e-ae4a-4edf-bdd9-87f4e3f51cbf None None] MFA Rules not processed for user `6cbc4765c42142a1aad99d6cc830e1b0`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:23.908597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f6ae2f6e-ae4a-4edf-bdd9-87f4e3f51cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:24.165817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1232/2474] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:24.182676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:24.228527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d8de62-1d3d-4ce9-b33a-8573db1447f0 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:24.238866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:24.240265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:24.241318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:24.244869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:24.412565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:24.414480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:24.434706 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1243/2475] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:24 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:25.804537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:25.805047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:25.805146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:25.836237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] MFA Rules not processed for user `7ba21f12574c4b9281d090651174788a`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:25.841005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:25.857544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1233/2476] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:24:25 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:26.862411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:26.878614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f3978-1931-40d9-9296-3692f0cd09e4 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:26.890571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:26.891279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:26.891654 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:26.892018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:26.894304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:26.914435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:26.916334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:26.936232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1244/2477] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:31.055783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:31.071074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8521adb8-fb86-4f26-9d29-3149d9b93da7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:31.076684 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:31.077287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:31.077563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:31.077831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:31.079591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:31.093814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:31.150128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:31.151768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea759d98-227a-4aa1-a10d-5ca5b5441a6f service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:31.161031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1234/2478] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:31 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:24:35.921280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:35.934491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a268f73-6cf1-4ab0-9fb0-7368123c5071 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:35.940031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:35.942035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:35.942470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:35.942615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:35.946464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:35.961082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:35.962809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:35.974536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1245/2479] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:35 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:41.156731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=21 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:43.648435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:43.649100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:43.649100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:43.670402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:24:43.671634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:43.757598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1235/2480] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:24:43 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:24:43.771452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:43.773017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:43.850147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:43.876533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c1b6e-ba4f-4825-9fca-8720db175d19 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:43.882187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:43.882913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:43.883261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:43.883757 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:43.894620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:43.895785 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:43.905630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1246/2481] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:24:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:24:44.019779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:44.047518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce1a5f-b48e-46c7-9a55-6a3b50c04543 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:44.071849 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:44.072581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:44.072937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:44.073280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:44.076337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:44.097868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:44.099834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14147df8-cc9b-43af-bb33-aef065a950cd service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:44.134286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1236/2482] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:44 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:45.187029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:24:45.215567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb991905-bdc9-49b6-b306-ec489423601a service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:24:45.223866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:24:45.224761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:24:45.225240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:24:45.225702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:24:45.231838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:24:45.265067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:24:45.266984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccfaca9-3e4a-4bd4-8f0c-a114b57db1f7 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:24:45.286485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1247/2483] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:24:45 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:24:54.107856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=31 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:24:55.280133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=18 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:00.627439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:00.629169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:00.630041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:00.632001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1237/2484] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:25:00 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:25:00.641181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e4b3a-26fb-4f31-a831-ed1d8260b559 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:00.642147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e4b3a-26fb-4f31-a831-ed1d8260b559 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:00.650597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466e4b3a-26fb-4f31-a831-ed1d8260b559 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:00.704905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-466e4b3a-26fb-4f31-a831-ed1d8260b559 None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:00.706492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-466e4b3a-26fb-4f31-a831-ed1d8260b559 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:00.737112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1248/2485] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:25:00 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:00.751904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:00.754144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:00.876357 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e041bdda-5261-4e1e-8909-4c6131d366f8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:00.879956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:00.880614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:00.880900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:00.881320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:00.894509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:00.896099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd1b6b03-02b7-4138-9bf1-87d048fb0953 service glance] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:00.909971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1238/2486] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:00 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:00.985910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:00.986194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:00.986574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] PATH_INFO: `/` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:00.987922 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1249/2487] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:25:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:25:00.992751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bec2-33ac-4af5-a3fc-f583d16c7bf9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:00.993051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bec2-33ac-4af5-a3fc-f583d16c7bf9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:00.993315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895bec2-33ac-4af5-a3fc-f583d16c7bf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:01.018735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a895bec2-33ac-4af5-a3fc-f583d16c7bf9 None None] MFA Rules not processed for user `5c3ad8bf2cc8457eb6685a514e545dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:01.020872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a895bec2-33ac-4af5-a3fc-f583d16c7bf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:01.033151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1239/2488] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:25:00 2022] POST /identity/v3/auth/tokens => generated 3121 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:01.049086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:01.051391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:01.072238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d174dc7-8927-4358-bd20-6baf3252c3fd None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:01.077380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:01.078198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:01.078697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:01.079182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:01.089709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:01.092018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f424ed12-b036-4c60-8bc9-da734381f802 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:01.102607 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1250/2489] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:25:01.199269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d36cf-b345-4fb5-ae18-f93dd820f6ae None None] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:01.199893 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d36cf-b345-4fb5-ae18-f93dd820f6ae None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:01.200278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817d36cf-b345-4fb5-ae18-f93dd820f6ae None None] PATH_INFO: `/` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:01.201856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1240/2490] 10.222.0.46 () {54 vars in 921 bytes} [Tue Aug 30 04:25:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Aug 30 04:25:01.207333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:01.207623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:01.207898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:01.228518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] MFA Rules not processed for user `148e575ef9b642c8b81d7eca8dd70bfa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:01.230220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:01.239841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1251/2491] 10.222.0.46 () {58 vars in 1018 bytes} [Tue Aug 30 04:25:01 2022] POST /identity/v3/auth/tokens => generated 3372 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:01.255557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d36cf-b345-4fb5-ae18-f93dd820f6ae None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:01.269268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817d36cf-b345-4fb5-ae18-f93dd820f6ae None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:01.273010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:01.273615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:01.273869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:01.274133 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:01.284883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:01.286310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:01.297485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1241/2492] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:01 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:25:03.249976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:03.259891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06d9a86-54ca-4a3a-804a-cfaec2a6d27d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:03.267663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:03.268122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:03.270444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:03.273598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:03.283787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:03.285317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b807b4d-a6de-414a-b5a0-9860737be465 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:03.298884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1252/2493] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:03 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:04.543842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:04.553163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d5876c-4c25-45e8-b209-2062767dbbf2 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:04.557142 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:04.559440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:04.559826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:04.560193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:04.562111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:04.637960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:04.639714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:04.651867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1242/2494] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:04.816723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:04.817590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:04.817862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:04.844855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:04.846661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:04.855868 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1253/2495] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:25:04 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:04.884115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:04.896645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f007213-17f1-470f-8beb-87cbf3b213dc service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:04.903784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:04.904617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:04.905019 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:04.905295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:04.914769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:04.916455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:04.926915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1243/2496] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:25:04 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:25:09.517855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:09.519647 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:09.535172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13a9b9a-1641-4b34-84a8-64ae7144d7d2 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:09.540059 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:09.540934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:09.541411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:09.542089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] PATH_INFO: `/v3/users/96d18f6efa1f4c91a6f213c095c507a3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:09.544196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96d18f6efa1f4c91a6f213c095c507a3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:09.546385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:09.596907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96d18f6efa1f4c91a6f213c095c507a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:09.613857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96d18f6efa1f4c91a6f213c095c507a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:09.614735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96d18f6efa1f4c91a6f213c095c507a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:09.615410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96d18f6efa1f4c91a6f213c095c507a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:09.629497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-346956dc-5147-4233-ab35-1d9e30be0ed5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96d18f6efa1f4c91a6f213c095c507a3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:09.643770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1254/2497] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:09 2022] DELETE /identity/v3/users/96d18f6efa1f4c91a6f213c095c507a3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:25:09.832578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:09.848586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b3138-a846-4b65-aa2f-75bbf5aa0f32 service swift] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:09.963447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:09.966699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:09.966981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:09.967286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] PATH_INFO: `/v3/projects/5a89b6f21ff24eafbed378c2ece79c3e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:09.972940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a89b6f21ff24eafbed378c2ece79c3e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:09.975977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:09.985782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:10.090755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a89b6f21ff24eafbed378c2ece79c3e', 'request_id': 'req-9db1e579-6e0c-474d-973c-4c2d3e2928b0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:10.109908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a89b6f21ff24eafbed378c2ece79c3e', 'request_id': 'req-9db1e579-6e0c-474d-973c-4c2d3e2928b0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:10.123650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a89b6f21ff24eafbed378c2ece79c3e', 'request_id': 'req-9db1e579-6e0c-474d-973c-4c2d3e2928b0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:10.140109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] The token cache is being invalidate because project 5a89b6f21ff24eafbed378c2ece79c3e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:10.140543 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:10.141621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9db1e579-6e0c-474d-973c-4c2d3e2928b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:10.145412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1244/2498] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:09 2022] DELETE /identity/v3/projects/5a89b6f21ff24eafbed378c2ece79c3e => generated 0 bytes in 313 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:10.149840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.150352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.150675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.173370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:10.174221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:10.243748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1255/2499] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:25:10.252235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.252512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.252826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.274612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:10.275844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:10.288746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1245/2500] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:10.293876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.304177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e234f96-530b-46a4-bcce-8cc261229f7a None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.309426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.310121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.310423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.310694 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.311348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:25:10.311725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.313260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.326302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1256/2501] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:25:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:25:10.331189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.342003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05d149a-765f-4ce9-837a-fb5c66b21c83 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.347493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.348118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.348392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.348669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.349376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.351431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.385685 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1246/2502] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/projects => generated 379 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:10.390417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.401102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678f0854-60ed-4f8a-8b5b-e1745e965ab8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.412110 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:10.474145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.474959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.475349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.475624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.476265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.478185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.503435 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1257/2503] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/users => generated 290 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:10.507560 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.519071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c14ec4-7dae-4f1a-a663-f617a23d6162 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.524148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.524876 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.525152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.525550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.526343 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.528792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.538943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1247/2504] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:10.543477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.554209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82033b40-7dd6-4306-ac03-2e7544cd970e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.560117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.560848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.561123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.561422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.562077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.563680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c79f64c-08a2-4854-9ac8-48722daa0459 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.576083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1258/2505] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:25:10.580327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.592342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ff9820-5348-40e0-821f-3083d683b19e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.597054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.597709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.597981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.598341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] PATH_INFO: `/v3/projects/88bfcb0f4b8943e98aa05999f3a17a8a/users/67ba68e6a015459f8e32842abb230dd6/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.616003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] RBAC: Authorizing `identity:create_grant(project_id=88bfcb0f4b8943e98aa05999f3a17a8a, user_id=67ba68e6a015459f8e32842abb230dd6, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.618333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.633817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1248/2506] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:10 2022] PUT /identity/v3/projects/88bfcb0f4b8943e98aa05999f3a17a8a/users/67ba68e6a015459f8e32842abb230dd6/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:10.647041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.647368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.647895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.682386 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] MFA Rules not processed for user `67ba68e6a015459f8e32842abb230dd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:10.684844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:10.763507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1259/2507] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:10.768411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.857772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4ac5d19-e93f-44cc-a1d1-e00c4865094b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.862434 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.863071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.863349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.863619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.864241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.866225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:10.902296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1249/2508] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/projects => generated 379 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:25:10.903797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:10.915065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-041799b4-8ed7-4226-8f77-3ccaddf6c623 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:10.975599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:10.976349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:10.976632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:10.976906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:10.977565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:10.980188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.008903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1260/2509] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:10 2022] POST /identity/v3/users => generated 296 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:25:11.013473 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.033508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bbb8c71-0a06-40ce-83ee-9166e60feb0a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.038726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.039301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.039605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.039872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.040515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.042213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.054152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1250/2510] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:11.058570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.070193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4b4e75-84d4-4980-a6d3-ce8a3287ac8e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.075493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.076304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.076761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.077043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] PATH_INFO: `/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2/users/b3a2f38d84764ea8ace00d6e57a85c69/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.087093 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:11.096593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=643dc8d79f2b490fbe5af7fdc35048e2, user_id=b3a2f38d84764ea8ace00d6e57a85c69, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.099391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.115725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1261/2511] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:11 2022] PUT /identity/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2/users/b3a2f38d84764ea8ace00d6e57a85c69/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:11.119934 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.126784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:11.131440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9310bbf-b120-4f54-b6ed-c0974c51b3d7 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.208409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.209125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.209464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.209810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.210569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.212377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1b6e43-afa8-40c1-ad21-411b997fa8f2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.233366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1251/2512] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:11 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:11.238071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.248767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f136534e-9ea5-4d0c-983e-d0f0e734caf2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.253586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.254276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.254571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.254845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] PATH_INFO: `/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2/users/b3a2f38d84764ea8ace00d6e57a85c69/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.256940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] RBAC: Authorizing `identity:create_grant(project_id=643dc8d79f2b490fbe5af7fdc35048e2, user_id=b3a2f38d84764ea8ace00d6e57a85c69, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.260249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.290735 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1262/2513] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:11 2022] PUT /identity/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2/users/b3a2f38d84764ea8ace00d6e57a85c69/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:11.302194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.302826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.303404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.343219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] MFA Rules not processed for user `b3a2f38d84764ea8ace00d6e57a85c69`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:11.345235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:11.438290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1252/2514] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:11 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:11.450813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.452550 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:11.518180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c19feba-e235-43dd-8fe4-0d53abf93b6f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.524145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.525577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.525577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.525577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.544241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.546731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08206f58-8659-4b8a-a976-5c85c161d00c service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.563390 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1263/2515] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:11 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:25:11.656368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:11.666566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455fa6b5-12d9-4d2b-a7d0-9c263d6d0a70 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:11.670917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:11.671617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:11.671955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:11.672286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:11.741540 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:11.742995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:11.752817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1253/2516] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:20.644990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:20.645490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:20.645668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:20.665725 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:20.667534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:20.724836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1264/2517] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:25:20 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:20.738090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:20.739651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:20.771414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:20.821920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92ead8e-79af-4073-bc3f-352a24f1da3e service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:20.828230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:20.828843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:20.829126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:20.829392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:20.838420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:20.840605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69117c1-1cae-496b-a799-23480af1f815 service swift] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:20.858127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1254/2518] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:25:20 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Aug 30 04:25:21.722086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:21.788280 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8037a81a-e45c-45be-bef1-ccca6d613295 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:21.792854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:21.793736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:21.794075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:21.794404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] PATH_INFO: `/v3/users/67ba68e6a015459f8e32842abb230dd6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:21.795683 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] RBAC: Authorizing `identity:delete_user(user_id=67ba68e6a015459f8e32842abb230dd6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:21.797834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:21.822827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:21.848908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67ba68e6a015459f8e32842abb230dd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:21.862417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67ba68e6a015459f8e32842abb230dd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:21.863131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67ba68e6a015459f8e32842abb230dd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:21.863763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67ba68e6a015459f8e32842abb230dd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:21.870748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67ba68e6a015459f8e32842abb230dd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:21.884638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1265/2519] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:21 2022] DELETE /identity/v3/users/67ba68e6a015459f8e32842abb230dd6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:21.889687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:21.890167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:21.890295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:21.909686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:21.910850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:21.978405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1255/2520] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:21.989130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:21.990665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:22.051265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2b25b2-df1c-43f6-82de-73e61764bdef admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:22.055343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:22.055960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:22.056240 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:22.056511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:22.066843 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:22.068275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:22.078123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1266/2521] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:21 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:22.646207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:22.647819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:22.661754 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394313db-fe3b-4d31-86d6-fc231a1b58b7 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:22.666693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:22.667322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:22.667699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:22.667885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] PATH_INFO: `/v3/projects/88bfcb0f4b8943e98aa05999f3a17a8a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:22.669161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=88bfcb0f4b8943e98aa05999f3a17a8a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:22.671026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:22.719007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '88bfcb0f4b8943e98aa05999f3a17a8a', 'request_id': 'req-bf5648c5-9d57-4821-a5f7-e8d38499a656'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.726289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '88bfcb0f4b8943e98aa05999f3a17a8a', 'request_id': 'req-bf5648c5-9d57-4821-a5f7-e8d38499a656'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.732592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '88bfcb0f4b8943e98aa05999f3a17a8a', 'request_id': 'req-bf5648c5-9d57-4821-a5f7-e8d38499a656'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.742720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] The token cache is being invalidate because project 88bfcb0f4b8943e98aa05999f3a17a8a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:22.743007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.743635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.745119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1256/2522] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:22 2022] DELETE /identity/v3/projects/88bfcb0f4b8943e98aa05999f3a17a8a => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:22.749236 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:22.758945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46edd-8454-4365-bd2c-f438ee514f91 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:22.817778 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:22.818801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:22.818801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:22.819120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] PATH_INFO: `/v3/users/b3a2f38d84764ea8ace00d6e57a85c69` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:22.820160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b3a2f38d84764ea8ace00d6e57a85c69)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:22.822088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:22.879786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:22.881374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b3a2f38d84764ea8ace00d6e57a85c69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.898431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b3a2f38d84764ea8ace00d6e57a85c69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.899259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b3a2f38d84764ea8ace00d6e57a85c69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.899787 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b3a2f38d84764ea8ace00d6e57a85c69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.908619 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bc0276c5-b76d-46c5-b6db-93638767a4b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b3a2f38d84764ea8ace00d6e57a85c69'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:22.923148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1267/2523] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:22 2022] DELETE /identity/v3/users/b3a2f38d84764ea8ace00d6e57a85c69 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:23.223394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.224837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:23.240425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5648c5-9d57-4821-a5f7-e8d38499a656 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.277611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:23.301887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.302614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.302888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.303152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] PATH_INFO: `/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.304325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] RBAC: Authorizing `identity:delete_project(project_id=643dc8d79f2b490fbe5af7fdc35048e2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.306213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.351853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '643dc8d79f2b490fbe5af7fdc35048e2', 'request_id': 'req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:23.359654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '643dc8d79f2b490fbe5af7fdc35048e2', 'request_id': 'req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:23.365968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '643dc8d79f2b490fbe5af7fdc35048e2', 'request_id': 'req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:23.377072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] The token cache is being invalidate because project 643dc8d79f2b490fbe5af7fdc35048e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:23.377358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:23.378028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a31cd2d4-13fc-4594-a32a-e0edef7ac85a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:23.379539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1257/2524] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:23 2022] DELETE /identity/v3/projects/643dc8d79f2b490fbe5af7fdc35048e2 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:25:23.385439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.385800 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.385982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.406653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:23.407776 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:23.465840 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1268/2525] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:23.473515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.473796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.474066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.494101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:23.495404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:23.503829 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1258/2526] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:23.508020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.517855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d883040-d552-4f69-a1d8-062eae74daf4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.522767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.523357 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.523636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.523907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.524549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:25:23.524851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.526456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.537876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1269/2527] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:25:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:25:23.542091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.551744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9edfc476-eb11-42f4-b2dd-336c0e5d649c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.556274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.556852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.557123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.557388 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.557999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.560566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.588258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1259/2528] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:23.593152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.605542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-500d1881-6795-4fe5-af14-45220f1df665 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.663136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.663674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.663952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.664223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.664862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.666793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.691707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1270/2529] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:23.696026 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.706606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-215ddd4b-2b85-49d0-bb71-4a21569c26fd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.711438 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.712021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.712302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.712572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.713211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.714821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.725324 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1260/2530] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:23.729219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.740109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb63eec-5684-4ec4-a9d6-858e5e9b14ff admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.744703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.745294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.745571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.745839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.746632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.748321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3ab0c6-3da5-4bfd-91d3-79e41258f53a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.758923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1271/2531] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:23 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:23.763420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.774548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6debfd28-1902-4faf-b53a-8d534a17d862 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:23.779698 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:23.780339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.780611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.780878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] PATH_INFO: `/v3/projects/f9ee128593a34f109abf2c23cec6b95a/users/4126d5e6ffca4d7a84c64eda57c3c68a/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.795554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] RBAC: Authorizing `identity:create_grant(project_id=f9ee128593a34f109abf2c23cec6b95a, user_id=4126d5e6ffca4d7a84c64eda57c3c68a, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:23.798163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:23.811326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1261/2532] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:23 2022] PUT /identity/v3/projects/f9ee128593a34f109abf2c23cec6b95a/users/4126d5e6ffca4d7a84c64eda57c3c68a/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:23.822818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:23.823102 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:23.823370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:23.851846 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] MFA Rules not processed for user `4126d5e6ffca4d7a84c64eda57c3c68a`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:23.853073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:23.870116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:23.919746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1272/2533] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:23.924298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:23.996120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33a9ec9c-80bd-4582-87cc-eb3ff631b41d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.001193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.001798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.002078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.002391 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.003047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.004941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.014600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:24.034295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1262/2534] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:23 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:24.038397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.049614 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301687a-1dd9-4238-b82d-a2f0db7abf7b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.107299 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.107907 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.108185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.108452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.109073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.111005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.131452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1273/2535] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:24 2022] POST /identity/v3/users => generated 299 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:24.135548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.147135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947659bc-35b4-4c19-8f49-b4c12f38db5a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.151744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.152336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.153238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.153508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.154144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.155798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.166029 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1263/2536] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:24.170633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.180571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ac8652b-4c7a-4e82-93d0-3e2099558bc4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.185244 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.185887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.186170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.186478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] PATH_INFO: `/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c/users/5e3014fa79424968822705fb9d401b79/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.202641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=77fbc7c03df64b9a9b86672ceeb0a46c, user_id=5e3014fa79424968822705fb9d401b79, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.205272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.219680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1274/2537] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:24 2022] PUT /identity/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c/users/5e3014fa79424968822705fb9d401b79/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:24.223638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.234460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81dfe03-bd6f-42d0-a6cf-c146f090f222 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.291894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.292498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.292777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.293050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.294151 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.295867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9686a3-12d0-4441-a0d0-261cd03d1403 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.305721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1264/2538] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:24.310358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.320641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c960082d-5565-4525-8bd5-6d3d46f970eb admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.325823 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.326560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.326839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.327109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] PATH_INFO: `/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c/users/5e3014fa79424968822705fb9d401b79/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.329008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=77fbc7c03df64b9a9b86672ceeb0a46c, user_id=5e3014fa79424968822705fb9d401b79, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.331648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.342407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1275/2539] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:24 2022] PUT /identity/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c/users/5e3014fa79424968822705fb9d401b79/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:25:24.352738 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.353016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.353284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.381624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] MFA Rules not processed for user `5e3014fa79424968822705fb9d401b79`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:24.382868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:24.457635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1265/2540] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:24 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:24.469413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.471158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:24.534399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e162276-93aa-4d03-b3ef-4bcfb9a917cd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.538037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.538661 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.538936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.539201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.548108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.549519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4f09d7-79c6-4370-aeec-52d4a4429889 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.559613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1276/2541] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:24 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:24.629416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.694021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc9c5891-1686-4cf4-9efc-956e67a095bb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.698853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.699534 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.699808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.700081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] PATH_INFO: `/v3/users/4126d5e6ffca4d7a84c64eda57c3c68a` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.701758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] RBAC: Authorizing `identity:delete_user(user_id=4126d5e6ffca4d7a84c64eda57c3c68a)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.703697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.705223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:24.765702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4126d5e6ffca4d7a84c64eda57c3c68a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:24.773079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4126d5e6ffca4d7a84c64eda57c3c68a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:24.779778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4126d5e6ffca4d7a84c64eda57c3c68a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:24.780393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4126d5e6ffca4d7a84c64eda57c3c68a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:24.792217 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4126d5e6ffca4d7a84c64eda57c3c68a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:24.798975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1266/2542] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:24 2022] DELETE /identity/v3/users/4126d5e6ffca4d7a84c64eda57c3c68a => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:24.803603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b681-af17-423c-8153-d62098add385 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.804120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b681-af17-423c-8153-d62098add385 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.804120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e802b681-af17-423c-8153-d62098add385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.825193 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e802b681-af17-423c-8153-d62098add385 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:24.826548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e802b681-af17-423c-8153-d62098add385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:24.834740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:24.888359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1277/2543] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:24 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:24.898750 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:24.900240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:24.956546 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ac7395-17f0-476f-8a84-03b92ab92dae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:24.960184 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:24.960711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:24.960980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:24.961241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:24.970117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:24.971576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:24.981600 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1267/2544] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:24 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:25.256122 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802b681-af17-423c-8153-d62098add385 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:25.257189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e802b681-af17-423c-8153-d62098add385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:25.274525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e802b681-af17-423c-8153-d62098add385 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:25.280577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:25.281209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:25.281533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:25.281736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] PATH_INFO: `/v3/projects/f9ee128593a34f109abf2c23cec6b95a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:25.282872 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f9ee128593a34f109abf2c23cec6b95a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:25.284782 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:25.330966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f9ee128593a34f109abf2c23cec6b95a', 'request_id': 'req-30bbad33-0069-41ed-a39a-1b9c814ad3c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.337132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f9ee128593a34f109abf2c23cec6b95a', 'request_id': 'req-30bbad33-0069-41ed-a39a-1b9c814ad3c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.345087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f9ee128593a34f109abf2c23cec6b95a', 'request_id': 'req-30bbad33-0069-41ed-a39a-1b9c814ad3c8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.355312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] The token cache is being invalidate because project f9ee128593a34f109abf2c23cec6b95a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:25.355668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.356896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.358375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1278/2545] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:25 2022] DELETE /identity/v3/projects/f9ee128593a34f109abf2c23cec6b95a => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:25:25.362444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:25.372652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b875e74-6fc8-4012-9b24-a0ca938393ab service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:25.429957 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:25.430637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:25.430920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:25.431192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] PATH_INFO: `/v3/users/5e3014fa79424968822705fb9d401b79` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:25.432355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e3014fa79424968822705fb9d401b79)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:25.434730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:25.458771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:25.485360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e3014fa79424968822705fb9d401b79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.492138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e3014fa79424968822705fb9d401b79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.499787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e3014fa79424968822705fb9d401b79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.500411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e3014fa79424968822705fb9d401b79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.512515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-7dfab094-0340-4279-a5b7-6f65f8442750 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e3014fa79424968822705fb9d401b79'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.518806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1268/2546] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:25 2022] DELETE /identity/v3/users/5e3014fa79424968822705fb9d401b79 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:25.795388 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:25.797397 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:25.835037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30bbad33-0069-41ed-a39a-1b9c814ad3c8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:25.893905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:25.894592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:25.894876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:25.895149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] PATH_INFO: `/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:25.896350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=77fbc7c03df64b9a9b86672ceeb0a46c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:25.898237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:25.938097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:25.946476 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '77fbc7c03df64b9a9b86672ceeb0a46c', 'request_id': 'req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.953120 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '77fbc7c03df64b9a9b86672ceeb0a46c', 'request_id': 'req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.960439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '77fbc7c03df64b9a9b86672ceeb0a46c', 'request_id': 'req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.970824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] The token cache is being invalidate because project 77fbc7c03df64b9a9b86672ceeb0a46c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:25.971116 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.971747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a8be7d17-a69c-47dc-97b4-cb03678a0a2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:25.973229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1279/2547] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:25 2022] DELETE /identity/v3/projects/77fbc7c03df64b9a9b86672ceeb0a46c => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:25.979262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:25.979541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:25.979807 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:25.999813 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:26.001032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:26.065630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1269/2548] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:25 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:26.075490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.075769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.076027 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.095861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:26.097077 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:26.105008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1280/2549] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:26 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:26.109509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.119121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c72da3-26b4-4653-9b83-8bc6b8da6add None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.124173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.124761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.125428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.125709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.126381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:25:26.126682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.128539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.140447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1270/2550] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:25:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:25:26.144924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.154239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4108ee4a-32c3-4530-8aa0-9434cfb77051 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.158941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.159535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.159812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.160081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.160703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.162655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.190412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1281/2551] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:26 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:26.194514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.205109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26767b45-dac9-4dcf-baba-36ee947fe3f9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.260933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.261544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.261844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.262711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.263366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.265488 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.287557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1271/2552] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:26 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:26.292008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.302900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea0b203-52e9-4dcb-86b6-4b76a6e0e1d5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.307680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.308278 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.308559 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.309352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.310226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.311877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.322136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1282/2553] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:26.327068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.336816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6638cfef-17ec-4d5c-88b5-f80c45f2780d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.341640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.342238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.342548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.342818 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.343457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.345087 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c555bcb0-8911-4195-8fbe-bca324973956 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.356872 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1272/2554] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:26 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:25:26.361188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.371959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60acc656-4e5c-42ad-a7a7-a6da6bbb954c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.378126 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.378859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.379139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.379412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] PATH_INFO: `/v3/projects/7b345fddd32442b39f7ca94f1f5f79f3/users/e8a44061557d4565be75a929eaeed3e4/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.394962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b345fddd32442b39f7ca94f1f5f79f3, user_id=e8a44061557d4565be75a929eaeed3e4, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.397620 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.410874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1283/2555] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:26 2022] PUT /identity/v3/projects/7b345fddd32442b39f7ca94f1f5f79f3/users/e8a44061557d4565be75a929eaeed3e4/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:26.422095 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.422617 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.422717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.436618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:26.452258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] MFA Rules not processed for user `e8a44061557d4565be75a929eaeed3e4`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:26.453471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:26.515556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1273/2556] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:26 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:26.526784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:26.528600 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:26.590189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b0c384-efaf-4235-890c-350ff50fbed1 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:26.593841 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:26.594484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:26.594766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:26.595036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:26.603483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:26.604899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a1c6b92-3592-4086-abf4-4cdf6d727d55 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:26.614360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1284/2557] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:26 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:29.202014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:29.267997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c810b660-9a60-4a8c-b1be-e46ebe7a44b2 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:29.274301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:29.274962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.275339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.275527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] PATH_INFO: `/v3/users/e8a44061557d4565be75a929eaeed3e4` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.277608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8a44061557d4565be75a929eaeed3e4)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:29.279719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:29.297713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.298305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.298305 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.319695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] MFA Rules not processed for user `545b18260abe47ceb440d0713439c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:29.320897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.333131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8a44061557d4565be75a929eaeed3e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.339831 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8a44061557d4565be75a929eaeed3e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.348372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8a44061557d4565be75a929eaeed3e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.349396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8a44061557d4565be75a929eaeed3e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.364455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-00ed91e7-e001-4d43-b9c2-3327243c6613 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8a44061557d4565be75a929eaeed3e4'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.370808 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1274/2558] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:29 2022] DELETE /identity/v3/users/e8a44061557d4565be75a929eaeed3e4 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:29.375789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.376077 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.376352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.379446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1285/2559] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:25:29.399059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:29.400249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.409522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:29.468352 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1275/2560] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:25:29.475257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:29.476669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.492067 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:29.540384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f1630b-114e-48ad-85c9-bb18bb24e3d5 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:29.591302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:29.591915 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.592196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.592483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.602113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:29.603568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:29.614167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1286/2561] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:29.696421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:29.698646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.711640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34c8a59-dd12-49f6-838c-6d7056cfe1fd None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:29.716368 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:29.717069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.717351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.717621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.719412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.729342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:29.730994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b4e729-0c3c-49b6-b062-27c7bb14ce39 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:29.739657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1276/2562] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:29 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:29.891081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:29.893505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:29.910405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbf909-703b-402c-906d-3e6b10fb9382 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:29.915281 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:29.915917 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:29.916200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:29.916618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] PATH_INFO: `/v3/projects/7b345fddd32442b39f7ca94f1f5f79f3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:29.918763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b345fddd32442b39f7ca94f1f5f79f3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:29.920956 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:29.974916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b345fddd32442b39f7ca94f1f5f79f3', 'request_id': 'req-9b017dda-fe37-4665-b1cb-00e8eb991395'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.982180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b345fddd32442b39f7ca94f1f5f79f3', 'request_id': 'req-9b017dda-fe37-4665-b1cb-00e8eb991395'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:29.990572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b345fddd32442b39f7ca94f1f5f79f3', 'request_id': 'req-9b017dda-fe37-4665-b1cb-00e8eb991395'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:30.002211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] The token cache is being invalidate because project 7b345fddd32442b39f7ca94f1f5f79f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:30.002626 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:30.003447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9b017dda-fe37-4665-b1cb-00e8eb991395 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:30.005682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1287/2563] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:29 2022] DELETE /identity/v3/projects/7b345fddd32442b39f7ca94f1f5f79f3 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:30.010703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.011016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.011291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.031764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:30.032959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:30.111508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1277/2564] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:30.119353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.119639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.119911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.141301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:30.142598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:30.151041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1288/2565] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:30.155889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.165574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3f59f5b-438b-4c10-a119-f42ff95deaca None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.170960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.171614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.171952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.172224 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.172866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:25:30.173163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.174888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.187422 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1278/2566] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:25:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:25:30.192256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.202565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ec7625-6e9f-4b6b-9957-9562233555a3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.207439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.208070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.208343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.208609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.209271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.211235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.239074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1289/2567] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:30 2022] POST /identity/v3/projects => generated 369 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:30.243381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.253944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3a33593-a3de-4185-9355-23d5e39a12b6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.315413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.316020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.316295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.316578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.317194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.319126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.341794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1279/2568] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:30 2022] POST /identity/v3/users => generated 285 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:25:30.346471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.357919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f50c88c3-5a00-4515-98a6-0c1c8d8254f4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.364053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.367976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.368410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.368690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.369413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.371188 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.386128 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1290/2569] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:30.388168 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.398773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e680010b-dcf1-4f4c-b8b4-595c20c92ee5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.403787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.404410 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.404678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.404948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.405612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.407248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401ccb4f-715d-4c9c-b8ad-f401dbd7f01c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.422125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1280/2570] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:30 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:25:30.423866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.435918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5990ff89-a698-494e-bf82-2a11dfc99183 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.441054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.441665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.441959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.442209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] PATH_INFO: `/v3/projects/0381c47d9974413890a335b59b88db01/users/f11a4b4a8c564359986fcf766eac896e/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.455035 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:30.459794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0381c47d9974413890a335b59b88db01, user_id=f11a4b4a8c564359986fcf766eac896e, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.462853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.478998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1291/2571] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:30 2022] PUT /identity/v3/projects/0381c47d9974413890a335b59b88db01/users/f11a4b4a8c564359986fcf766eac896e/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:25:30.490730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.491079 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.491354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.521066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] MFA Rules not processed for user `f11a4b4a8c564359986fcf766eac896e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:30.522080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:30.523085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:30.594353 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1281/2572] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:30.607113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:30.608873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:30.670835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1885676-02f2-4e15-bbd9-475309c96d49 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:30.674887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '288a7a6690d3480db235d2f594bc0acd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:30.675525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:30.675808 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:30.676083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:30.687292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:30.688722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc328cb-803d-4d87-9bff-91701bb8c6cc service glance] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:30.698082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1292/2573] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:30 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:40.590716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=28 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:40.693827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:57.520185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:57.582732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d416d2-5196-4fb1-8d1b-ff0f6f742e2f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:57.587326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:57.587953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:57.588232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:57.588503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] PATH_INFO: `/v3/users/f11a4b4a8c564359986fcf766eac896e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:57.590390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f11a4b4a8c564359986fcf766eac896e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:57.592392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:57.639221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f11a4b4a8c564359986fcf766eac896e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:57.646154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f11a4b4a8c564359986fcf766eac896e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:57.654031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f11a4b4a8c564359986fcf766eac896e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:57.654753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f11a4b4a8c564359986fcf766eac896e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:57.666271 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f11a4b4a8c564359986fcf766eac896e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:57.672211 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1282/2574] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:25:57 2022] DELETE /identity/v3/users/f11a4b4a8c564359986fcf766eac896e => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:57.676773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:57.677269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:57.677384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:57.696710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:57.697905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:57.756664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1293/2575] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:57 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:57.766999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:57.768761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:57.827215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9e01f5-4249-4f9d-bce8-d5795ba79f11 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:57.831012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:57.831616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:57.831889 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:57.832154 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:57.841515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:57.842980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84898c3f-25dc-4e9d-a166-bf974508908e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:57.852417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1283/2576] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:57 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:25:58.121810 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.123318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:58.136547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04214a68-7bb5-4c8c-beb4-e128332d1595 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.141270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.142412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.142693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.142963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] PATH_INFO: `/v3/projects/0381c47d9974413890a335b59b88db01` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.144351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0381c47d9974413890a335b59b88db01)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.146237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.201805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0381c47d9974413890a335b59b88db01', 'request_id': 'req-d30ccfcb-7439-4bc4-b018-42129bb53699'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:58.209667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0381c47d9974413890a335b59b88db01', 'request_id': 'req-d30ccfcb-7439-4bc4-b018-42129bb53699'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:58.219261 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0381c47d9974413890a335b59b88db01', 'request_id': 'req-d30ccfcb-7439-4bc4-b018-42129bb53699'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:58.232239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] The token cache is being invalidate because project 0381c47d9974413890a335b59b88db01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:25:58.232558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:58.233220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d30ccfcb-7439-4bc4-b018-42129bb53699 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:25:58.234836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1294/2577] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:25:58 2022] DELETE /identity/v3/projects/0381c47d9974413890a335b59b88db01 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:58.239845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.240129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.240404 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.263528 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:58.264911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:58.290112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:58.335389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1284/2578] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:58.343741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.344029 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.344303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.363900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:58.365108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:58.374184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1295/2579] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:58 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:58.378521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.389003 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e416a1-5550-47f0-91dc-2686e9a13cd0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.393906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.394529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.394804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.395074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.395715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:25:58.396014 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.397603 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.409610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1285/2580] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:25:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:25:58.413914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.423172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b78653-2676-4bd6-ab91-c97dd1238705 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.427856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.428437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.428715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.428986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.429623 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.431531 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.459172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1296/2581] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:25:58 2022] POST /identity/v3/projects => generated 391 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:58.463440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.473580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c12dd3-246f-4612-a24f-6af4a3d88c2c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.530599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.531287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.531455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.531719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.532339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.534226 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.555814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1286/2582] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:25:58 2022] POST /identity/v3/users => generated 296 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:25:58.560801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.576817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69e982c-0a30-43db-b917-78c0884b45c8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.584393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.585002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.585287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.585615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.586207 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.587875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.598596 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1297/2583] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:58.602716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.613591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3563a9a-31e3-4a21-a47b-bd831104f748 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.618164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.618780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.619049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.619313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.619946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.622531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857d497d-ff7a-45f3-b0de-f3155dfd08fb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.632783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1287/2584] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:25:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:25:58.636966 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.647651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b291fae-d4eb-4037-af23-573ab69a4b96 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.652341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.653086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.653371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.653644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] PATH_INFO: `/v3/projects/7bee6dda73184cd9a19ccb482d72f4ea/users/ac405a06bcce424d921e8c4851282cd6/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.668773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bee6dda73184cd9a19ccb482d72f4ea, user_id=ac405a06bcce424d921e8c4851282cd6, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.671580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.684834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1298/2585] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:25:58 2022] PUT /identity/v3/projects/7bee6dda73184cd9a19ccb482d72f4ea/users/ac405a06bcce424d921e8c4851282cd6/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:25:58.696199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.696479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.696746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.729798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] MFA Rules not processed for user `ac405a06bcce424d921e8c4851282cd6`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:25:58.731549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:58.793768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1288/2586] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:25:58 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:25:58.804652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:25:58.806387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:25:58.828715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:25:58.864995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900e07c5-6aa4-4095-8d1c-582f061198fa admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:25:58.868713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:25:58.869322 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:25:58.869592 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:25:58.869862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:25:58.878982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:25:58.880268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:25:58.888957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1299/2587] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:25:58 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:08.790401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:08.884878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=17 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:16.175483 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:16.176917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:16.239074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d300efa-49f3-4cc7-9dee-584e7460517d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:16.242667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:16.243262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:16.243533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:16.243797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:16.245482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:16.309702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:16.311157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42903d82-36ec-4963-b124-37935868b80d service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:16.320824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1289/2588] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:16 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:26:18.271169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:18.286197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc760da1-5333-40d7-ac26-9819d8a2f20b service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:18.291944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:18.294448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:18.294811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:18.295185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] PATH_INFO: `/v3/users/ac405a06bcce424d921e8c4851282cd6` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:18.297358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac405a06bcce424d921e8c4851282cd6)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:18.299939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:18.362407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac405a06bcce424d921e8c4851282cd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.376220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac405a06bcce424d921e8c4851282cd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.377349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac405a06bcce424d921e8c4851282cd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.377901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac405a06bcce424d921e8c4851282cd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.385972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac405a06bcce424d921e8c4851282cd6'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.400223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1300/2589] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:18 2022] DELETE /identity/v3/users/ac405a06bcce424d921e8c4851282cd6 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:18.404929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:18.405498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:18.405498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:18.425860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:18.427099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:18.497612 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1290/2590] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:18 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:18.507491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:18.508964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:18.569971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec681dd-d26a-43d5-8877-7b7d92202728 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:18.573546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:18.574393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:18.574878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:18.574878 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:18.584039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:18.585452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:18.595859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1301/2591] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:18 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:18.900462 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:18.902795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:18.916083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334c6146-5b88-42b7-b093-c6e9027edf54 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:18.921469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:18.922121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:18.922441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:18.922715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] PATH_INFO: `/v3/projects/7bee6dda73184cd9a19ccb482d72f4ea` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:18.923929 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7bee6dda73184cd9a19ccb482d72f4ea)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:18.925917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:18.981680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:18.987532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7bee6dda73184cd9a19ccb482d72f4ea', 'request_id': 'req-9e290978-9222-4202-8ffa-1b1b1c8b2a17'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:18.998529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7bee6dda73184cd9a19ccb482d72f4ea', 'request_id': 'req-9e290978-9222-4202-8ffa-1b1b1c8b2a17'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.005997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7bee6dda73184cd9a19ccb482d72f4ea', 'request_id': 'req-9e290978-9222-4202-8ffa-1b1b1c8b2a17'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.024383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] The token cache is being invalidate because project 7bee6dda73184cd9a19ccb482d72f4ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:19.024749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.025632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.028105 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1291/2592] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:18 2022] DELETE /identity/v3/projects/7bee6dda73184cd9a19ccb482d72f4ea => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:19.823708 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:19.825321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:19.838609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d980bf76-7301-49ea-92be-920c2739e8fe service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:19.897939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:19.898724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:19.899073 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:19.899350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] PATH_INFO: `/v3/users/8c30ec9810204a4685590c42707e41f8` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:19.900580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c30ec9810204a4685590c42707e41f8)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:19.902514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:19.920217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:19.962502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c30ec9810204a4685590c42707e41f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.976231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c30ec9810204a4685590c42707e41f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.977242 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c30ec9810204a4685590c42707e41f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.977856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c30ec9810204a4685590c42707e41f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:19.988549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-005022b3-a4e9-4e7c-b201-01c4400ed7b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c30ec9810204a4685590c42707e41f8'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.002575 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1302/2593] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:19 2022] DELETE /identity/v3/users/8c30ec9810204a4685590c42707e41f8 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:20.175572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.177053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:20.192897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e290978-9222-4202-8ffa-1b1b1c8b2a17 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.251331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.251958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.252323 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.252601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] PATH_INFO: `/v3/projects/411b8da28bed456bbdcdb01587610b47` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.253751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] RBAC: Authorizing `identity:delete_project(project_id=411b8da28bed456bbdcdb01587610b47)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.255690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.305963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '411b8da28bed456bbdcdb01587610b47', 'request_id': 'req-607af787-c1a2-4c96-b01c-e7b3b375f03c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.314327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '411b8da28bed456bbdcdb01587610b47', 'request_id': 'req-607af787-c1a2-4c96-b01c-e7b3b375f03c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.320827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '411b8da28bed456bbdcdb01587610b47', 'request_id': 'req-607af787-c1a2-4c96-b01c-e7b3b375f03c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.333068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] The token cache is being invalidate because project 411b8da28bed456bbdcdb01587610b47 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:20.333355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.334016 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-607af787-c1a2-4c96-b01c-e7b3b375f03c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:20.335539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1292/2594] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:20 2022] DELETE /identity/v3/projects/411b8da28bed456bbdcdb01587610b47 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:20.341447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.341724 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.342016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.365304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:20.366564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:20.427423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1303/2595] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:26:20.435679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.435977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.436229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.456345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:20.457476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:20.466059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1293/2596] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:20.470364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.480161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6669b8b7-22ef-4892-b222-95f402b59631 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.484713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.485304 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.485584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.485856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.486855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:20.487157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.488738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.501016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1304/2597] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:20.505282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.515051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8992620e-b2a8-4ac2-b848-f983bfc40954 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.520031 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.520616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.520892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.521155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.521786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.523703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.550764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1294/2598] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:20 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:20.554935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.565085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6cdd8c-b343-4e57-b076-2250132b9b68 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.621695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.622332 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.622618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.622894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.623523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.625419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.648237 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1305/2599] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:20 2022] POST /identity/v3/users => generated 292 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:20.653173 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.664201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9835146-0993-4294-a897-a457dc8d69cd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.669377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.669980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.670277 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.670551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.671196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.672798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.682979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1295/2600] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:20.687282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.697555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98de3768-8f6d-4fe1-92be-751d1aabda39 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.703113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.703716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.703996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.704269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.704910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.706590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bae583a5-eec6-441e-af12-926902292144 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.712064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=9,keystone:SELECT=87,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:20.719415 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1306/2601] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:20 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:20.723787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.734311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52d6ed9-c178-4ec5-9532-3d45bc8eac0d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.739174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.739819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.740093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.740359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] PATH_INFO: `/v3/projects/18aa4b5d08004df59401773d97b1f545/users/b2825f64b838452cbc1c962802757439/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.755517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] RBAC: Authorizing `identity:create_grant(project_id=18aa4b5d08004df59401773d97b1f545, user_id=b2825f64b838452cbc1c962802757439, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.758186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.768798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:20.774907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1296/2602] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:20 2022] PUT /identity/v3/projects/18aa4b5d08004df59401773d97b1f545/users/b2825f64b838452cbc1c962802757439/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:26:20.785554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.785842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.786130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.815807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] MFA Rules not processed for user `b2825f64b838452cbc1c962802757439`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:20.817008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:20.881598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1307/2603] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:20 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:20.893212 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:20.895194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:20.967552 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb12893-4f5e-4bea-9a57-7b352504d68d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:20.973020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:20.973611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:20.973886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:20.974153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:20.982216 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:20.983838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe39366-293f-4f9a-8f4a-c42d0038d970 service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:20.993307 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1297/2604] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:20 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:28.285844 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:28.348558 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f056178-f08e-4069-9d71-e0a0b1a5dcaa None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:28.353062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:28.353693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:28.353970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:28.354239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] PATH_INFO: `/v3/users/b2825f64b838452cbc1c962802757439` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:28.355941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2825f64b838452cbc1c962802757439)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:28.357839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:28.404852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2825f64b838452cbc1c962802757439'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.416454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2825f64b838452cbc1c962802757439'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.417051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2825f64b838452cbc1c962802757439'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.418208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2825f64b838452cbc1c962802757439'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.425481 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2825f64b838452cbc1c962802757439'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.437869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1308/2605] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:28 2022] DELETE /identity/v3/users/b2825f64b838452cbc1c962802757439 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:28.442411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:28.442886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:28.442983 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:28.462059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:28.463288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:28.521955 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1298/2606] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:28.532315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:28.534535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:28.546693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:28.596272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5838ab-1aa8-4bdf-b332-1a26bed4ddbc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:28.600028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:28.600637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:28.600921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:28.601195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:28.611395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:28.612809 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd75a7e-c4c7-4078-8894-616ff2a2ddd4 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:28.623018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1309/2607] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:28 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:28.894981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:28.896629 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:28.909787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c9e95f1-f142-4592-8a47-2ea6d89d10af None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:28.915032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:28.916032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:28.916315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:28.916588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] PATH_INFO: `/v3/projects/18aa4b5d08004df59401773d97b1f545` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:28.917905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=18aa4b5d08004df59401773d97b1f545)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:28.920022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:28.968042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18aa4b5d08004df59401773d97b1f545', 'request_id': 'req-b1a56857-e7e6-456e-ab6d-b6c16679a13a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.975325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18aa4b5d08004df59401773d97b1f545', 'request_id': 'req-b1a56857-e7e6-456e-ab6d-b6c16679a13a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.982337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18aa4b5d08004df59401773d97b1f545', 'request_id': 'req-b1a56857-e7e6-456e-ab6d-b6c16679a13a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.992726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] The token cache is being invalidate because project 18aa4b5d08004df59401773d97b1f545 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:28.993121 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.993776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b1a56857-e7e6-456e-ab6d-b6c16679a13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:28.995321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1299/2608] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:28 2022] DELETE /identity/v3/projects/18aa4b5d08004df59401773d97b1f545 => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:29.001169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.001457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.001729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.021729 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:29.022989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:29.082456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1310/2609] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:29.091542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.091826 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.092097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.112770 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:29.114034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:29.123295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1300/2610] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:29 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:29.127568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.139233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-896f4ce5-b7e0-4199-a7e2-f966044f090e None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.144105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.144711 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.144978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.145243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.146121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:29.146282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.147799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.163774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1311/2611] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:29.168069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.172801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:29.179608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7bffb8-81bd-451b-9760-5d1871161863 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.186763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.187371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.187679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.187980 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.188794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.190744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.219905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1301/2612] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:29 2022] POST /identity/v3/projects => generated 375 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:29.224037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.234727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfb0344-660d-41f8-862d-12f4a72bd27c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.296904 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.297513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.297794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.298115 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.298791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.301147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.323243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1312/2613] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:29 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:29.328122 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.339013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ec2048-3187-43c0-ad5e-534640048b5f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.344179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.344782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.345061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.345333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.345975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.347605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.366067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1302/2614] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:29.367404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.382389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df812aab-72da-4f0b-83ac-db5efb718e7c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.388893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.389643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.389924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.390185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.393393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.395417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92cbbb24-4d3a-4de5-a33e-3b9e5c8c664d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.415412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1313/2615] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:29 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:29.421302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.432999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07b78bc7-3d1b-4ead-ba0d-48b1d6b702eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.438988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.440170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.440752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.441110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] PATH_INFO: `/v3/projects/49a91297c79848ecb786f3c1c75a7930/users/882eba86d1c34c158f69f562fe127ffa/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.457686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a91297c79848ecb786f3c1c75a7930, user_id=882eba86d1c34c158f69f562fe127ffa, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.460339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.478291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1303/2616] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:29 2022] PUT /identity/v3/projects/49a91297c79848ecb786f3c1c75a7930/users/882eba86d1c34c158f69f562fe127ffa/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:29.485572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.485858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.486131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.515194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] MFA Rules not processed for user `882eba86d1c34c158f69f562fe127ffa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:29.516519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:29.521901 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:29.591150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1314/2617] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:29 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:29.602089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.603802 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:29.687330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2cebf93-23f8-4ac4-8a27-3ec5c0006560 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.692950 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.693555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.693836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.694110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.702455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:29.704043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-374ce47a-d53a-41c5-a028-b005a826b57e service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:29.712941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1304/2618] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:29 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:29.929740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:29.991301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eecc68c1-4fe5-4185-85ab-13fe088103ba None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:29.995812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:29.996697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:29.996980 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:29.997249 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] PATH_INFO: `/v3/users/882eba86d1c34c158f69f562fe127ffa` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:29.998454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=882eba86d1c34c158f69f562fe127ffa)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.000353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.055285 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '882eba86d1c34c158f69f562fe127ffa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.067090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '882eba86d1c34c158f69f562fe127ffa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.067762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '882eba86d1c34c158f69f562fe127ffa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.068375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '882eba86d1c34c158f69f562fe127ffa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.075133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '882eba86d1c34c158f69f562fe127ffa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.087170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1315/2619] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:29 2022] DELETE /identity/v3/users/882eba86d1c34c158f69f562fe127ffa => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:30.092414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.092701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.092969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.113558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:30.114805 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:30.173457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1305/2620] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:30.183055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.184597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:30.216296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:30.243926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec15e27-aaaf-4f98-b8f6-acae1a283d01 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.247675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.248272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.248549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.248815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.257631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.259057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9f7d6a-09f4-49ad-806a-b2311a6b61f2 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.270076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1316/2621] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:30 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:30.541459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.542964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:30.555853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bac8efe-dd5c-4910-a00a-16955243d02d None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.561412 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.562047 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.562351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.562632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] PATH_INFO: `/v3/projects/49a91297c79848ecb786f3c1c75a7930` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.563757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a91297c79848ecb786f3c1c75a7930)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.565651 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.600640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=2,keystone:DELETE=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:30.619656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a91297c79848ecb786f3c1c75a7930', 'request_id': 'req-6f512573-4599-4026-bca5-fc41542afc7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.628408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a91297c79848ecb786f3c1c75a7930', 'request_id': 'req-6f512573-4599-4026-bca5-fc41542afc7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.635101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a91297c79848ecb786f3c1c75a7930', 'request_id': 'req-6f512573-4599-4026-bca5-fc41542afc7b'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.647295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] The token cache is being invalidate because project 49a91297c79848ecb786f3c1c75a7930 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:30.647590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.648270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6f512573-4599-4026-bca5-fc41542afc7b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:30.650103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1306/2622] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:30 2022] DELETE /identity/v3/projects/49a91297c79848ecb786f3c1c75a7930 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:30.655019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.655525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.655622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.675767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:30.676996 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:30.741459 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1317/2623] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:30.749855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.750136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.750779 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.772365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:30.773664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:30.782174 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1307/2624] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:30.786562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.796565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c31d2b7-0daa-4aae-bffc-09f661dada14 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.801111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.801709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.801988 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.802275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.802927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:30.803231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.804812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.819194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1318/2625] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:26:30.823548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.833672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d93af20-9c7b-40a3-a38e-ba0cbd91bcff None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.838327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.838922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.839193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.839523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.840349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.842232 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.869953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1308/2626] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:30 2022] POST /identity/v3/projects => generated 405 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:30.875091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.885200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471d51ba-5ca3-46d2-af5f-ac8b1720751b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.948190 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.948838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.949117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.949395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.950030 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.952047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:30.976344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1319/2627] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:30 2022] POST /identity/v3/users => generated 303 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:30.977693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:30.989356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c964ca26-2f26-4ca2-baef-e715663783bd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:30.993938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:30.994596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:30.994871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:30.995134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:30.996327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:30.997942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.008021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1309/2628] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:31.012713 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.023339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68bca8f-e613-425b-b90e-ad1d2855b179 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.028613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.029213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.029489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.029755 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.030424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.031998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730bed39-d6fb-4482-abc8-d2cd973f8f97 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.045419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1320/2629] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:31.049377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.061379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d97731ad-bac9-4333-b65a-2da061a62d44 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.066644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.067338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.067614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.067883 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] PATH_INFO: `/v3/projects/ee182cbe30ce43958d2d5e7871ab7d1f/users/efba987c268b4c5b936b7f4fb49e11a7/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.083440 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ee182cbe30ce43958d2d5e7871ab7d1f, user_id=efba987c268b4c5b936b7f4fb49e11a7, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.086075 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.101749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1310/2630] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:31 2022] PUT /identity/v3/projects/ee182cbe30ce43958d2d5e7871ab7d1f/users/efba987c268b4c5b936b7f4fb49e11a7/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:31.114838 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.115121 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.115392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.145864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] MFA Rules not processed for user `efba987c268b4c5b936b7f4fb49e11a7`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:31.147164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:31.172275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:31.213496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1321/2631] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3282 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:26:31.218513 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.283115 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03645f6f-1864-4513-bcee-6b465ee175c8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.288004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.288609 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.288887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.289160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.289796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.291735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.319628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1311/2632] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:31 2022] POST /identity/v3/projects => generated 405 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:31.324274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.335572 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58191761-9e1e-4531-a966-3fa507850c22 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.398556 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.399179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.399463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.399899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.400508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.403064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.433853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1322/2633] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:31 2022] POST /identity/v3/users => generated 309 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:31.437730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.448251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2125b2fc-3603-4938-a423-98a4b68a0bae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.453290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.453890 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.454169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.456241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.456241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.456721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.468524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1312/2634] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:26:31.472695 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.483949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259832e-3e05-46ad-89cf-7401234df6a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.488727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.489392 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.489677 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.489951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] PATH_INFO: `/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31/users/3a24a66d444d4caab227e7ab31ce27e1/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.507400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e033c25e13e42ad92e55fd8d69b7d31, user_id=3a24a66d444d4caab227e7ab31ce27e1, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.510129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.524884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1323/2635] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:31 2022] PUT /identity/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31/users/3a24a66d444d4caab227e7ab31ce27e1/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:31.529267 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.539621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3879c3-a29a-4517-b560-ae7a7af5c259 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.560575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:31.605744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.606485 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.606787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.607057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.607727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.609336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89270d70-5a10-445a-be24-eb7da8d06105 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.621674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1313/2636] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:31 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:31.626449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.639068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5fb5b0-3869-46bc-82da-d1e3f07ec272 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.646422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.646422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.646422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.646422 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] PATH_INFO: `/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31/users/3a24a66d444d4caab227e7ab31ce27e1/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.650686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9e033c25e13e42ad92e55fd8d69b7d31, user_id=3a24a66d444d4caab227e7ab31ce27e1, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.653446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:31.665308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1324/2637] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:31 2022] PUT /identity/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31/users/3a24a66d444d4caab227e7ab31ce27e1/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:31.677613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.678366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.678366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.724104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] MFA Rules not processed for user `3a24a66d444d4caab227e7ab31ce27e1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:31.728884 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:31.836344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1314/2638] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:31 2022] POST /identity/v3/auth/tokens => generated 3349 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:31.848343 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:31.850421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:31.959998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee517eb-8bf1-4d3e-a8f0-a6f326e38481 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:31.965043 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:31.966414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:31.966866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:31.967591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:31.986473 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:31.988031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-693b30c9-65cd-4d5d-a3ce-ae9d20472117 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:32.002170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1325/2639] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:31 2022] GET /identity/v3/auth/tokens => generated 3349 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:26:32.145397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:32.209366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed34165f-c8a3-4bd5-a5f0-0cfdc1bb9c48 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:32.213958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:32.215251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:32.215567 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:32.215885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] PATH_INFO: `/v3/users/efba987c268b4c5b936b7f4fb49e11a7` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:32.217032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=efba987c268b4c5b936b7f4fb49e11a7)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:32.218994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:32.276384 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'efba987c268b4c5b936b7f4fb49e11a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.279256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:32.290777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'efba987c268b4c5b936b7f4fb49e11a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.300304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'efba987c268b4c5b936b7f4fb49e11a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.300990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'efba987c268b4c5b936b7f4fb49e11a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.316720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'efba987c268b4c5b936b7f4fb49e11a7'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.326341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1315/2640] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:32 2022] DELETE /identity/v3/users/efba987c268b4c5b936b7f4fb49e11a7 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:32.329625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:32.329910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:32.330181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:32.362109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:32.363854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:32.380547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:32.437314 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1326/2641] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:32 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:32.447661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:32.449362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:32.510804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6b8783-8cdc-4436-b3a4-3d6283d7bf63 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:32.514982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:32.515152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:32.515345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:32.515611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:32.525056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:32.526524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:32.536380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1316/2642] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:32 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:32.820031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:32.821216 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:32.842581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b049bf54-e1ed-4ec5-8682-a5fd2ef83b4d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:32.848944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:32.849578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:32.849896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:32.850269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] PATH_INFO: `/v3/projects/ee182cbe30ce43958d2d5e7871ab7d1f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:32.851447 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ee182cbe30ce43958d2d5e7871ab7d1f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:32.853546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:32.914834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ee182cbe30ce43958d2d5e7871ab7d1f', 'request_id': 'req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.922039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ee182cbe30ce43958d2d5e7871ab7d1f', 'request_id': 'req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.930929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ee182cbe30ce43958d2d5e7871ab7d1f', 'request_id': 'req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.943039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] The token cache is being invalidate because project ee182cbe30ce43958d2d5e7871ab7d1f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:32.943329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.943990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:32.945483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1327/2643] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:32 2022] DELETE /identity/v3/projects/ee182cbe30ce43958d2d5e7871ab7d1f => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:32.949476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:32.959371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95330dde-faaa-4e94-a743-821b3056f231 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:33.045220 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:33.046186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.046648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.047021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] PATH_INFO: `/v3/users/3a24a66d444d4caab227e7ab31ce27e1` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.048444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a24a66d444d4caab227e7ab31ce27e1)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:33.053982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:33.111601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a24a66d444d4caab227e7ab31ce27e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.118678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a24a66d444d4caab227e7ab31ce27e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.132138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a24a66d444d4caab227e7ab31ce27e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.132982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a24a66d444d4caab227e7ab31ce27e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.141409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:INSERT=2,keystone:DELETE=12 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:33.151744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-52dd3602-745a-4aea-a309-f0125300de85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a24a66d444d4caab227e7ab31ce27e1'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.159913 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1317/2644] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:32 2022] DELETE /identity/v3/users/3a24a66d444d4caab227e7ab31ce27e1 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:33.471174 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:33.474450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:33.494938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbd2d5f-0732-4f1e-9e6f-d3ab8ad0f520 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:33.561545 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:33.562323 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.562606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.562887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] PATH_INFO: `/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.564033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9e033c25e13e42ad92e55fd8d69b7d31)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:33.566114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:33.616053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:33.619068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9e033c25e13e42ad92e55fd8d69b7d31', 'request_id': 'req-3c287062-62fb-4f1f-b262-eb03fd427e65'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.627279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9e033c25e13e42ad92e55fd8d69b7d31', 'request_id': 'req-3c287062-62fb-4f1f-b262-eb03fd427e65'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.636795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9e033c25e13e42ad92e55fd8d69b7d31', 'request_id': 'req-3c287062-62fb-4f1f-b262-eb03fd427e65'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.654822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] The token cache is being invalidate because project 9e033c25e13e42ad92e55fd8d69b7d31 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:33.655108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.655763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-3c287062-62fb-4f1f-b262-eb03fd427e65 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:33.657471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1328/2645] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:33 2022] DELETE /identity/v3/projects/9e033c25e13e42ad92e55fd8d69b7d31 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:33.662413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.662915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.663191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.684197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:33.685406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:33.750266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1318/2646] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:33.758354 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.758641 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.758965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.780603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:33.781827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:33.789963 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1329/2647] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:33 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:33.794580 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:33.805417 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a7de2ff-06a4-4b5d-9949-86239eaa3445 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:33.815148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:33.815720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.816002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.816278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.816935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:33.817241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:33.819170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:33.831726 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1319/2648] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:33.836053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:33.846010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d280b7-068f-4d5b-a476-b76452b8c910 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:33.850865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:33.851474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.851759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.852032 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.852664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:33.854555 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:33.882710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1330/2649] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:33 2022] POST /identity/v3/projects => generated 357 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:33.887660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:33.898299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7223f1ca-ef5e-463d-89f4-397682447f13 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:33.964021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:33.964896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:33.965447 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:33.965693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:33.966395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:33.968308 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:33.991172 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1320/2650] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:33 2022] POST /identity/v3/users => generated 279 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:26:33.995208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.005885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a591fc2-63f6-4e2d-abb9-6f5c04471f9a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.015264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.015879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.016157 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.016424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.017065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.018962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.032320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1331/2651] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:34.033605 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.044387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b41afc99-2602-4313-a73d-8e15e05422d6 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.049269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.049798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.050072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.050405 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.051050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.052665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf5d4f2-550a-495d-955d-3c87447c105b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.064524 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1321/2652] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:34.068761 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.078705 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd98bda-ddfa-4fb0-b055-b30b1ab53cf3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.083395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.084025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.084301 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.084570 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] PATH_INFO: `/v3/projects/aed4cb771fbc4be595dea73fda8b256b/users/22b2c2e7e91a4d528c275a3afd68fe9b/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.099087 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aed4cb771fbc4be595dea73fda8b256b, user_id=22b2c2e7e91a4d528c275a3afd68fe9b, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.101691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.115152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1332/2653] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:34 2022] PUT /identity/v3/projects/aed4cb771fbc4be595dea73fda8b256b/users/22b2c2e7e91a4d528c275a3afd68fe9b/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:26:34.127298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.127582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.127877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.155809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] MFA Rules not processed for user `22b2c2e7e91a4d528c275a3afd68fe9b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:34.156958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:34.199763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:DELETE=1,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:34.224443 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1322/2654] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:34 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:26:34.228884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.290062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed27dad3-4627-4899-8345-237b9feec0f2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.295262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.295853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.296130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.296395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.297026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.298945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.324997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1333/2655] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:34 2022] POST /identity/v3/projects => generated 355 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:34.329318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.340515 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf372e3d-0b66-4a80-9bb0-56bf2fa0cfc9 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.396946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.397550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.397832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.398102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.398765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.400683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.423119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1323/2656] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:34 2022] POST /identity/v3/users => generated 284 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:34.427584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.439100 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d06f2fbb-d72e-4318-bb64-fae010a01ff7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.443827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.444417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.444696 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.445259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.446002 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.447727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.458209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1334/2657] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:26:34.462423 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.473100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053954fe-e153-4d22-a406-64c1b58abaf9 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.477735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.478647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.478922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.479188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] PATH_INFO: `/v3/projects/c36c21201f0040e5b6708bee90d1a3bc/users/ebe1691f88eb4b5491aec1bd64459729/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.494209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36c21201f0040e5b6708bee90d1a3bc, user_id=ebe1691f88eb4b5491aec1bd64459729, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.497135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.510096 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1324/2658] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:34 2022] PUT /identity/v3/projects/c36c21201f0040e5b6708bee90d1a3bc/users/ebe1691f88eb4b5491aec1bd64459729/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:34.514381 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.524784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33981786-75da-4a35-a382-5cdf3fe1a4df admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.567210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:DELETE=4,keystone:SELECT=92,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:34.584069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.584671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.584955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.585223 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.585853 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.587456 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45c2cba-5624-4115-9fc1-d40eb7045103 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.597798 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1335/2659] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:34 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:34.601876 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.612731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe15d8a3-afee-4d31-ab08-b940469564ca admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.617339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.617973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.618272 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.618539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] PATH_INFO: `/v3/projects/c36c21201f0040e5b6708bee90d1a3bc/users/ebe1691f88eb4b5491aec1bd64459729/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.620433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36c21201f0040e5b6708bee90d1a3bc, user_id=ebe1691f88eb4b5491aec1bd64459729, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.623338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.633376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1325/2660] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:34 2022] PUT /identity/v3/projects/c36c21201f0040e5b6708bee90d1a3bc/users/ebe1691f88eb4b5491aec1bd64459729/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:34.644442 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.644726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.645000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.674665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] MFA Rules not processed for user `ebe1691f88eb4b5491aec1bd64459729`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:34.675549 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:34.742998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1336/2661] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:34.754666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.756409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:34.815716 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a6676d-b4c2-48fa-9a11-e21e2517a6eb admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.819507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.820112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.820390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.820665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.831199 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.832644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-028abc78-4f0a-424e-992b-be5dc17615c7 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:34.842906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1326/2662] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:34 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:34.919265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:34.929544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350640e0-8327-4f3c-b13d-6e9ced6fa119 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:34.934182 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:34.934925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:34.935306 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:34.935667 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:34.993023 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:34.994496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:35.004038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1337/2663] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:34 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:44.837925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:44.999883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=71 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:46.067646 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67eaa-785a-4f4e-995a-5f3b47671a6e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:46.068164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67eaa-785a-4f4e-995a-5f3b47671a6e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:46.068164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c67eaa-785a-4f4e-995a-5f3b47671a6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:46.087240 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-01c67eaa-785a-4f4e-995a-5f3b47671a6e None None] MFA Rules not processed for user `545b18260abe47ceb440d0713439c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:46.088453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-01c67eaa-785a-4f4e-995a-5f3b47671a6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:46.144094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1327/2664] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:46 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:50.162646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:50.228991 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc70f0f-877f-47f1-a647-8384e4529df0 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:50.234081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:50.234812 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:50.235094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:50.235359 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] PATH_INFO: `/v3/users/22b2c2e7e91a4d528c275a3afd68fe9b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:50.236815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=22b2c2e7e91a4d528c275a3afd68fe9b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:50.238451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:50.296355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '22b2c2e7e91a4d528c275a3afd68fe9b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:50.308797 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '22b2c2e7e91a4d528c275a3afd68fe9b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:50.309490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '22b2c2e7e91a4d528c275a3afd68fe9b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:50.310143 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '22b2c2e7e91a4d528c275a3afd68fe9b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:50.317794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '22b2c2e7e91a4d528c275a3afd68fe9b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:50.334004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1338/2665] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:50 2022] DELETE /identity/v3/users/22b2c2e7e91a4d528c275a3afd68fe9b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:50.338471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:50.338941 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:50.339036 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:50.360940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:50.362157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:50.426966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1328/2666] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:50 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:50.436931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:50.438409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:50.498528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41e6adb-fb29-4e58-9f43-78fb875da0a3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:50.502142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:50.502799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:50.503083 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:50.503353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:50.513045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:50.514490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:50.524246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1339/2667] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:50 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:50.939878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:50.941359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:50.954409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba44457-8e24-47ec-a26c-435f29e2ad7c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:50.965481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:50.966136 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:50.966449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:50.966709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] PATH_INFO: `/v3/projects/aed4cb771fbc4be595dea73fda8b256b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:50.968290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=aed4cb771fbc4be595dea73fda8b256b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:50.971116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.022787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aed4cb771fbc4be595dea73fda8b256b', 'request_id': 'req-de8cfff3-c04a-4194-a291-6f017e46f0ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.031897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aed4cb771fbc4be595dea73fda8b256b', 'request_id': 'req-de8cfff3-c04a-4194-a291-6f017e46f0ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.038504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aed4cb771fbc4be595dea73fda8b256b', 'request_id': 'req-de8cfff3-c04a-4194-a291-6f017e46f0ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.049355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] The token cache is being invalidate because project aed4cb771fbc4be595dea73fda8b256b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:51.049355 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.050144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.051681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1329/2668] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:50 2022] DELETE /identity/v3/projects/aed4cb771fbc4be595dea73fda8b256b => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:51.055803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:51.069112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55f71561-e3ae-44da-855d-662ca2df287a service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:51.088968 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:51.130547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:51.131180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.131562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.131835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] PATH_INFO: `/v3/users/ebe1691f88eb4b5491aec1bd64459729` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.132959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] RBAC: Authorizing `identity:delete_user(user_id=ebe1691f88eb4b5491aec1bd64459729)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:51.134887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.182445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ebe1691f88eb4b5491aec1bd64459729'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.196882 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ebe1691f88eb4b5491aec1bd64459729'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.197608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ebe1691f88eb4b5491aec1bd64459729'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.198220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ebe1691f88eb4b5491aec1bd64459729'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.207181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9979c85e-2982-412b-b7bb-682fe113a1fc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ebe1691f88eb4b5491aec1bd64459729'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.219938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1340/2669] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:51 2022] DELETE /identity/v3/users/ebe1691f88eb4b5491aec1bd64459729 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:51.524254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:51.525746 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:51.539843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8cfff3-c04a-4194-a291-6f017e46f0ed admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:51.567846 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:51.602902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:51.603547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.603824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.604097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] PATH_INFO: `/v3/projects/c36c21201f0040e5b6708bee90d1a3bc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.605949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36c21201f0040e5b6708bee90d1a3bc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:51.608040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.654764 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36c21201f0040e5b6708bee90d1a3bc', 'request_id': 'req-9df78550-553e-40d5-99e1-61f8df59d074'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.662020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36c21201f0040e5b6708bee90d1a3bc', 'request_id': 'req-9df78550-553e-40d5-99e1-61f8df59d074'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.668987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36c21201f0040e5b6708bee90d1a3bc', 'request_id': 'req-9df78550-553e-40d5-99e1-61f8df59d074'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.679377 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] The token cache is being invalidate because project c36c21201f0040e5b6708bee90d1a3bc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:51.679647 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.680286 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-9df78550-553e-40d5-99e1-61f8df59d074 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:51.681827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1330/2670] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:51 2022] DELETE /identity/v3/projects/c36c21201f0040e5b6708bee90d1a3bc => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:51.686743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.687034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.687313 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.706640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:51.707852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:51.769507 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1341/2671] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:51.779366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.779578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.779857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.800635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:51.801852 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:51.810878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1331/2672] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:51.815427 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:51.825589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d7c8c8-fc9e-4ef1-ba9b-e925da26b29b None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:51.830201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:51.830833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.831136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.831406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.832075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:51.832374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:51.833953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.846048 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1342/2673] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:51.850406 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:51.860788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7ffe3a-fa53-4ff1-9de9-25fde431edc8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:51.865457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:51.866050 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.866362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.866632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.867291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:51.869185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.898638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1332/2674] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:51 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:51.902967 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:51.913016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-903deac4-8669-4605-9556-36dbd3bee9f3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:51.940097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:51.972891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:51.973493 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:51.973770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:51.974038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:51.974733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:51.976784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:51.998815 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1343/2675] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:51 2022] POST /identity/v3/users => generated 295 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:52.003013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.013330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768f704a-62f3-40ba-864d-5709587d02db admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.017906 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.018526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.019057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.019326 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.020392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.021979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.032262 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1333/2676] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:52.036052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.047074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0bf09c-e1f7-478f-a0b6-6a94effb9a0e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.051747 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.052341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.052625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.052920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.053553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.055169 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d18e46-248d-4a93-950d-6da6a3666e28 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.066702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1344/2677] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:52.071058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.081563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fc4c68f-0058-4ab0-8a98-ee43d7809899 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.086215 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.086882 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.087161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.087430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] PATH_INFO: `/v3/projects/678a48f3c96c4d93b21f8a80895b00a5/users/fd509f1a6beb4282947963a7843f35a9/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.101842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=678a48f3c96c4d93b21f8a80895b00a5, user_id=fd509f1a6beb4282947963a7843f35a9, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.104578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.118005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1334/2678] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:52 2022] PUT /identity/v3/projects/678a48f3c96c4d93b21f8a80895b00a5/users/fd509f1a6beb4282947963a7843f35a9/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:52.130941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.131224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.131518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.159873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] MFA Rules not processed for user `fd509f1a6beb4282947963a7843f35a9`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:52.161089 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:52.226341 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1345/2679] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:52 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:52.231611 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.289696 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:52.296556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba94427-55bd-4ff9-871a-c91b2d580e27 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.302135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.302773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.303055 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.303354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.303982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.305888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.333847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1335/2680] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:52 2022] POST /identity/v3/projects => generated 387 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:26:52.337911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.350467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb94901-c84e-41eb-9960-eeabe3051796 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.407777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.408418 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.408698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.408971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.409601 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.411564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.433197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1346/2681] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:52 2022] POST /identity/v3/users => generated 300 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:26:52.437426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.447971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c42537-4eb3-4650-8722-d5509f4923fd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.452652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.453273 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.453548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.453820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.454482 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.456160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.467146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1336/2682] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:52.471308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.482230 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4639039e-7e77-41cb-beb8-d88f192367c5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.486811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.487461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.487741 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.488010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] PATH_INFO: `/v3/projects/c290531f422d46bd8526c9c6acd0c605/users/fa664651c0b2464fb85dcd4832c26c68/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.503638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290531f422d46bd8526c9c6acd0c605, user_id=fa664651c0b2464fb85dcd4832c26c68, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.506277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.520857 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1347/2683] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:52 2022] PUT /identity/v3/projects/c290531f422d46bd8526c9c6acd0c605/users/fa664651c0b2464fb85dcd4832c26c68/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:52.524999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.536776 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307aa732-5519-4d9b-a6c6-6ea304a0e689 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.602652 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.603305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.603631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.603956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.604724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.607241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccdf6b37-9789-49a0-9d72-6718737f782c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.623971 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1337/2684] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:52 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:52.625181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.630676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:52.637378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2add513-b6e6-483d-b058-e3a2858161a9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.642709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.643369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.643911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.644187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] PATH_INFO: `/v3/projects/c290531f422d46bd8526c9c6acd0c605/users/fa664651c0b2464fb85dcd4832c26c68/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.646138 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c290531f422d46bd8526c9c6acd0c605, user_id=fa664651c0b2464fb85dcd4832c26c68, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.648777 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.660135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1348/2685] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:52 2022] PUT /identity/v3/projects/c290531f422d46bd8526c9c6acd0c605/users/fa664651c0b2464fb85dcd4832c26c68/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:52.672061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.672345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.672650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.702235 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] MFA Rules not processed for user `fa664651c0b2464fb85dcd4832c26c68`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:52.703469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:52.769126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1338/2686] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:52 2022] POST /identity/v3/auth/tokens => generated 3331 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:26:52.780943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:52.782743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:52.843209 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcb6693-c0f1-4059-89d1-0cec7b9d9256 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:52.846916 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:52.847516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:52.847799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:52.848466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:52.857505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:52.858971 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9bad8f9-baa6-4f11-beb3-58e239bee88f service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:52.868225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1349/2687] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:52 2022] GET /identity/v3/auth/tokens => generated 3331 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:53.006021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:53.055480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:53.074338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea2aac0-3a93-40bc-82ed-3fd23ec8b16f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:53.078922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:53.079585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:53.079869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:53.080150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] PATH_INFO: `/v3/users/fd509f1a6beb4282947963a7843f35a9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:53.081328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fd509f1a6beb4282947963a7843f35a9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:53.083294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:53.131433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fd509f1a6beb4282947963a7843f35a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.139005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fd509f1a6beb4282947963a7843f35a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.147303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fd509f1a6beb4282947963a7843f35a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.147966 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fd509f1a6beb4282947963a7843f35a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.160590 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fd509f1a6beb4282947963a7843f35a9'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.166760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1339/2688] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:53 2022] DELETE /identity/v3/users/fd509f1a6beb4282947963a7843f35a9 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:53.171960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:53.172325 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:53.172691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:53.193129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:53.194471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:53.256645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1350/2689] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:53 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:53.267201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:53.268952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:53.329894 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a646db-c66e-4375-b2e5-e3526586d8b8 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:53.334239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:53.335002 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:53.335287 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:53.335557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:53.346372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:53.347822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:53.358837 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1340/2690] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:53 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:53.631542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:53.633130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:53.647687 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777e9f88-e0c7-4934-ba1f-62eb34ebe0d1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:53.653463 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:53.654134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:53.654455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:53.654727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] PATH_INFO: `/v3/projects/678a48f3c96c4d93b21f8a80895b00a5` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:53.655867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] RBAC: Authorizing `identity:delete_project(project_id=678a48f3c96c4d93b21f8a80895b00a5)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:53.657768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:53.717276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '678a48f3c96c4d93b21f8a80895b00a5', 'request_id': 'req-fdc4cbce-1c26-4871-a678-33f4164c5a45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.723383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '678a48f3c96c4d93b21f8a80895b00a5', 'request_id': 'req-fdc4cbce-1c26-4871-a678-33f4164c5a45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.731133 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '678a48f3c96c4d93b21f8a80895b00a5', 'request_id': 'req-fdc4cbce-1c26-4871-a678-33f4164c5a45'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.742139 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] The token cache is being invalidate because project 678a48f3c96c4d93b21f8a80895b00a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:53.742452 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.743081 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.745055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1351/2691] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:53 2022] DELETE /identity/v3/projects/678a48f3c96c4d93b21f8a80895b00a5 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:53.749591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:53.759457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebd4ea5-0e7a-47fe-b659-5d06c45913bf service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:53.828597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:53.829230 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:53.829504 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:53.829772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] PATH_INFO: `/v3/users/fa664651c0b2464fb85dcd4832c26c68` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:53.831007 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa664651c0b2464fb85dcd4832c26c68)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:53.832900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:53.847541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:53.882947 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa664651c0b2464fb85dcd4832c26c68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.890616 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa664651c0b2464fb85dcd4832c26c68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.899256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa664651c0b2464fb85dcd4832c26c68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.899943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa664651c0b2464fb85dcd4832c26c68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.911940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4183a45a-8713-485b-bf8a-c564be04fdde admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa664651c0b2464fb85dcd4832c26c68'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:53.917724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1341/2692] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:53 2022] DELETE /identity/v3/users/fa664651c0b2464fb85dcd4832c26c68 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:54.188820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.190707 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:54.210462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:54.280214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc4cbce-1c26-4871-a678-33f4164c5a45 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.335692 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.336320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.336599 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.336870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] PATH_INFO: `/v3/projects/c290531f422d46bd8526c9c6acd0c605` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.338016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c290531f422d46bd8526c9c6acd0c605)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.340202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.386130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c290531f422d46bd8526c9c6acd0c605', 'request_id': 'req-99bbc173-0d1c-447b-830c-9fa92b68b808'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:54.392715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c290531f422d46bd8526c9c6acd0c605', 'request_id': 'req-99bbc173-0d1c-447b-830c-9fa92b68b808'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:54.400698 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c290531f422d46bd8526c9c6acd0c605', 'request_id': 'req-99bbc173-0d1c-447b-830c-9fa92b68b808'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:54.411591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] The token cache is being invalidate because project c290531f422d46bd8526c9c6acd0c605 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:54.411879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:54.412513 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-99bbc173-0d1c-447b-830c-9fa92b68b808 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:54.414060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1352/2693] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:54 2022] DELETE /identity/v3/projects/c290531f422d46bd8526c9c6acd0c605 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:54.419551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.419962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.420336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.478608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:54.479794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:54.543496 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1342/2694] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:54.551353 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.551639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.551913 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.571666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:54.572903 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:54.581145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1353/2695] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:54 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:54.585572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.595860 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3617acb1-86df-466d-8919-b7ece1e88d3f None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.600503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.601092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.601371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.601644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.602317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:54.602622 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.604223 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.616463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1343/2696] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:54.620890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.630449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f476fda3-c9e7-4bd6-93d6-1c622ce93239 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.635161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.635751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.636106 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.636554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.637407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.639339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.668129 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1354/2697] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:54 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:54.672246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.682459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-847f6207-1638-440a-ae5f-47d2c91b5a01 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.726790 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:54.741108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.741636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.741909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.742176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.742824 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.744711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.766760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1344/2698] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:54 2022] POST /identity/v3/users => generated 286 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:54.771134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.782108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84b4554-3dbe-45f8-812c-7b686cb963c2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.786790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.787379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.787655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.787920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.788551 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.790109 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.800465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1355/2699] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:54.804476 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.814549 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f35144f-f961-4787-a84d-f9945a7ebf40 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.819245 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.819841 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.820119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.820381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.821011 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.822624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9990c5a-4368-4d35-aa35-0f4cf900e7a4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.834453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1345/2700] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:54 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:54.838786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:54.845653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:54.849643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ee5873-49e6-4130-80a7-477f2113a365 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:54.856243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:54.857028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.857467 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.857792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] PATH_INFO: `/v3/projects/14958c0f883a43069b72f38b77be97a9/users/d27f8ce415d3484799b029c6a6c1c99e/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.873919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=14958c0f883a43069b72f38b77be97a9, user_id=d27f8ce415d3484799b029c6a6c1c99e, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:54.875912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:54.890276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1356/2701] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:54 2022] PUT /identity/v3/projects/14958c0f883a43069b72f38b77be97a9/users/d27f8ce415d3484799b029c6a6c1c99e/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:54.901018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:54.901298 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:54.901564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:54.930327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] MFA Rules not processed for user `d27f8ce415d3484799b029c6a6c1c99e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:54.931582 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:54.999697 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1346/2702] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:54 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:55.012200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:55.013947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:55.088071 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f08af7-8672-46d4-a4c2-84f29dd31ec8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:55.095145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:55.096152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.096478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.096928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.109180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:55.110674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e931f-7236-477b-a009-5b2dc092ca5b service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:55.123352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1357/2703] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:55 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:55.145642 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:55.215254 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e374acbe-b378-48c3-8a66-7e2675c03017 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:55.219995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:55.220630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.220907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.221178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] PATH_INFO: `/v3/users/d27f8ce415d3484799b029c6a6c1c99e` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.222366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d27f8ce415d3484799b029c6a6c1c99e)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:55.224288 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:55.274394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd27f8ce415d3484799b029c6a6c1c99e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.283321 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd27f8ce415d3484799b029c6a6c1c99e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.286371 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:55.291810 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd27f8ce415d3484799b029c6a6c1c99e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.292918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd27f8ce415d3484799b029c6a6c1c99e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.308437 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd27f8ce415d3484799b029c6a6c1c99e'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.314905 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1347/2704] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:26:55 2022] DELETE /identity/v3/users/d27f8ce415d3484799b029c6a6c1c99e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:55.319512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.320333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.320608 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.341891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:55.343259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:55.421629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1358/2705] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:55.431517 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:55.433559 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:55.500193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568567f5-2bab-4440-86d4-e8c9e5d3a258 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:55.504104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:55.504729 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.505005 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.505278 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.515655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:55.517027 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09050775-32f8-41f0-9589-7d655a710220 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:55.527531 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1348/2706] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:55 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:26:55.860329 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:55.861909 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:55.874565 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b4d9c3-5865-4db2-b37c-905589a67165 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:55.879527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:55.880246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.880590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.880959 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] PATH_INFO: `/v3/projects/14958c0f883a43069b72f38b77be97a9` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.882189 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=14958c0f883a43069b72f38b77be97a9)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:55.884203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:55.932511 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '14958c0f883a43069b72f38b77be97a9', 'request_id': 'req-c6672ed7-8d73-4417-aada-b0ea7c207b8c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.938557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '14958c0f883a43069b72f38b77be97a9', 'request_id': 'req-c6672ed7-8d73-4417-aada-b0ea7c207b8c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.946902 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '14958c0f883a43069b72f38b77be97a9', 'request_id': 'req-c6672ed7-8d73-4417-aada-b0ea7c207b8c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.949469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:55.959737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] The token cache is being invalidate because project 14958c0f883a43069b72f38b77be97a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:26:55.960022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.960675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-c6672ed7-8d73-4417-aada-b0ea7c207b8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:26:55.962168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1359/2707] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:26:55 2022] DELETE /identity/v3/projects/14958c0f883a43069b72f38b77be97a9 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:55.967834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:55.968295 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:55.968411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:55.988548 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:55.989755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:56.049024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1349/2708] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:55 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:56.057801 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.058611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.058987 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.079374 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:56.080681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:56.088752 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1360/2709] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:56.093275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.103085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a518ba2a-dd38-41e8-9456-25c3fe140988 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.107863 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.108471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.108756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.109025 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.109675 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:26:56.109979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.111623 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.124161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1350/2710] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:26:56.128591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.138873 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46646a8f-8259-4d58-a06c-c3c871a5fce3 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.143598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.144455 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.144734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.145003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.145842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.147759 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.183552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1361/2711] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:56.184769 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.194922 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7f6ad5-3bc7-42fd-a570-58d1b01b9ac5 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.252919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.253526 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.253801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.254071 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.254731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.256710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.269498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:56.282063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1351/2712] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/users => generated 292 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:26:56.286235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.296790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97b02097-84a5-46c2-b778-a052b2d7ab6e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.301496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.302103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.302409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.302680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.303320 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.304900 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.316200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1362/2713] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:56.377261 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.388072 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c54af8e-a428-4184-a8ec-7799528274dc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.392712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.393309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.393584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.394108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.394772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.396491 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76038958-3750-4f65-bc6e-aa0c6802cf44 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.413402 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1352/2714] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:56.414548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.425270 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00e29535-1888-44ef-8562-52c0bd1fc0c2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.430927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.431688 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.431973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.432246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] PATH_INFO: `/v3/projects/194aaad51f8f4463b9d648ab9ae9a3c2/users/7ba577f017e0462693e348766064cfdc/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.449462 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=194aaad51f8f4463b9d648ab9ae9a3c2, user_id=7ba577f017e0462693e348766064cfdc, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.452656 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.466034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1363/2715] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:56 2022] PUT /identity/v3/projects/194aaad51f8f4463b9d648ab9ae9a3c2/users/7ba577f017e0462693e348766064cfdc/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:26:56.475938 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.476249 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.476672 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.507960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] MFA Rules not processed for user `7ba577f017e0462693e348766064cfdc`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:56.509171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:56.574985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1353/2716] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:56.579430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.644297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64213a63-369d-46f8-867d-a2d928ddbbf2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.649003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.649613 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.649888 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.650158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.650817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.653238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.681957 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1364/2717] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:56.683564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.694129 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ba7360-a480-4a19-9b87-4955f640a279 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.750131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.750758 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.751034 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.751305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.752090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.753832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.775667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1354/2718] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:56.780283 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.791131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1273d31-434d-402d-a5cd-558c40942f21 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.796014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.796609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.796889 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.797187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.797824 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.799448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.810892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1365/2719] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:56.815322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.826379 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ef54ca-93f5-4150-b282-4caeacdc8943 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.831065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.831664 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.831940 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.832209 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.832848 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.834466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a9eba6-2859-4a3e-b26d-261b7549b3a3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.845420 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1355/2720] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:56.849540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:56.860407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5790d04-4c18-4d30-874f-e04a2d62f670 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:56.865474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:56.866137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.866446 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.866721 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] PATH_INFO: `/v3/projects/32f739d9490d4bd485d2a89848a9f36d/users/3449706317c64e7293f49e343a033a47/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.883195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] RBAC: Authorizing `identity:create_grant(project_id=32f739d9490d4bd485d2a89848a9f36d, user_id=3449706317c64e7293f49e343a033a47, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:56.885876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:56.900340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1366/2721] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:56 2022] PUT /identity/v3/projects/32f739d9490d4bd485d2a89848a9f36d/users/3449706317c64e7293f49e343a033a47/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:56.910927 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:56.911290 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:56.911633 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:56.941309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] MFA Rules not processed for user `3449706317c64e7293f49e343a033a47`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:56.942654 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:56.985789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:57.015426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1356/2722] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:56 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:57.016748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.069373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:DELETE=1,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:57.093790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106b2574-4c60-4f89-a9bc-a59f35a5f30d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.099852 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.100573 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.101897 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.102037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.102634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.105509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.138504 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1367/2723] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:26:57 2022] POST /identity/v3/projects => generated 385 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:57.140427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.153594 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efda5d6d-1293-4903-802c-a5d3cb6c5e35 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.218225 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.218856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.219134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.219398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.220048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.222104 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.248258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1357/2724] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:26:57 2022] POST /identity/v3/users => generated 299 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:26:57.252508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.264918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bb03228-97e8-43a8-9de1-c203e89ce4f6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.270906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.272038 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.272179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.272489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.273457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.275722 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.289172 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1368/2725] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:26:57.294394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.306525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46581222-e326-4a0d-924b-8001160ee26e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.311961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.312686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.313028 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.313364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] PATH_INFO: `/v3/projects/f8888425c0844e49b92d3b65d1f7591f/users/375ad328c80b4646b3a20190675df574/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.328875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8888425c0844e49b92d3b65d1f7591f, user_id=375ad328c80b4646b3a20190675df574, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.331477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.345191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1358/2726] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:57 2022] PUT /identity/v3/projects/f8888425c0844e49b92d3b65d1f7591f/users/375ad328c80b4646b3a20190675df574/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:26:57.349181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.360063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2693f33-4b28-4f5a-9c84-36635a229489 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.423610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.424723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.424723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.424921 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.425526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.427198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f46aa8c-2bd1-4809-a028-960bb239645e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.440197 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1369/2727] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:26:57 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:26:57.443431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.455318 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cb17bb2-8785-47ab-876d-9435c3fbcef4 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.460397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.461048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.461327 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.461634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] PATH_INFO: `/v3/projects/f8888425c0844e49b92d3b65d1f7591f/users/375ad328c80b4646b3a20190675df574/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.463588 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f8888425c0844e49b92d3b65d1f7591f, user_id=375ad328c80b4646b3a20190675df574, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.467241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.478139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1359/2728] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:26:57 2022] PUT /identity/v3/projects/f8888425c0844e49b92d3b65d1f7591f/users/375ad328c80b4646b3a20190675df574/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:26:57.490726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.491015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.491316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.520790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] MFA Rules not processed for user `375ad328c80b4646b3a20190675df574`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:26:57.522008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:57.597871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1370/2729] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:26:57 2022] POST /identity/v3/auth/tokens => generated 3329 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:26:57.610994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:57.612778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:57.671635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9351867-57cc-42f0-96de-cce0d0b9a0a3 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:57.675536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:57.676180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:57.676456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:57.676833 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:57.725821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:57.737686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:57.739463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:57.750964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1360/2730] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:59.295130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:59.297264 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:26:59.310221 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44455180-2bb0-4a15-a1c3-662fd996793c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:59.314041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:59.314611 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:59.314884 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:59.315148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:59.348147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:26:59.389222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:59.390703 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:59.400321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1371/2731] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:59 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:26:59.951653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:26:59.960398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a175b859-ff36-41a6-9ab6-864f8865ed93 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:26:59.964522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:26:59.965167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:26:59.965448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:26:59.965721 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:26:59.974733 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:26:59.976143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:26:59.986164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1361/2732] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:26:59 2022] GET /identity/v3/auth/tokens => generated 3329 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:03.532569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:03.533955 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:03.624879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd62e929-f6fc-48c7-ab37-c236347cf946 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:03.630328 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:03.630931 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:03.631208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:03.631610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:03.633279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:03.645669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:03.647629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab71f5eb-091d-4779-b56c-b5aab2ef88ba service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:03.658142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1372/2733] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:03 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:03.889677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:03.962081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7defad2a-d2f4-45ad-a1cb-64efe2e15740 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:03.967493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:03.968118 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:03.968396 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:03.968666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] PATH_INFO: `/v3/users/7ba577f017e0462693e348766064cfdc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:03.969794 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba577f017e0462693e348766064cfdc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:03.971732 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:04.018411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba577f017e0462693e348766064cfdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.025407 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba577f017e0462693e348766064cfdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.032492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba577f017e0462693e348766064cfdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.033311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba577f017e0462693e348766064cfdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.046356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba577f017e0462693e348766064cfdc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.052167 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1362/2734] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:27:03 2022] DELETE /identity/v3/users/7ba577f017e0462693e348766064cfdc => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:04.056628 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:04.056912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:04.057202 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:04.076450 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:04.077643 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:04.136470 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1373/2735] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:04 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:04.146745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:04.148233 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:04.208341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed236683-595c-4c6c-ba63-96822c2d6d02 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:04.212069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:04.212671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:04.212948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:04.213238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:04.224351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:04.225840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:04.230283 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:04.239660 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1363/2736] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:04 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:04.532593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:04.534152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:04.546131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3390a7-ef22-40a5-b45c-4dead29cdf10 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:04.550826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:04.551464 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:04.551744 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:04.552014 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] PATH_INFO: `/v3/projects/194aaad51f8f4463b9d648ab9ae9a3c2` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:04.553136 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] RBAC: Authorizing `identity:delete_project(project_id=194aaad51f8f4463b9d648ab9ae9a3c2)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:04.555063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:04.595338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:04.608218 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '194aaad51f8f4463b9d648ab9ae9a3c2', 'request_id': 'req-aaa9ee16-e243-4faf-b0ef-16e650420e34'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.614311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '194aaad51f8f4463b9d648ab9ae9a3c2', 'request_id': 'req-aaa9ee16-e243-4faf-b0ef-16e650420e34'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.623052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '194aaad51f8f4463b9d648ab9ae9a3c2', 'request_id': 'req-aaa9ee16-e243-4faf-b0ef-16e650420e34'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.633479 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] The token cache is being invalidate because project 194aaad51f8f4463b9d648ab9ae9a3c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:27:04.633768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.634429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.635942 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1374/2737] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:27:04 2022] DELETE /identity/v3/projects/194aaad51f8f4463b9d648ab9ae9a3c2 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:04.639935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:04.650722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6565cc-988e-4b4d-8559-ed2cb68dbf23 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:04.707778 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:04.708439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:04.708719 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:04.708990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] PATH_INFO: `/v3/users/3449706317c64e7293f49e343a033a47` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:04.710166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3449706317c64e7293f49e343a033a47)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:04.712190 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:04.758238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3449706317c64e7293f49e343a033a47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.765679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3449706317c64e7293f49e343a033a47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.774201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3449706317c64e7293f49e343a033a47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.774997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3449706317c64e7293f49e343a033a47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.787453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3449706317c64e7293f49e343a033a47'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:04.794053 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1364/2738] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:27:04 2022] DELETE /identity/v3/users/3449706317c64e7293f49e343a033a47 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:05.066433 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:05.067927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:05.082439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaa9ee16-e243-4faf-b0ef-16e650420e34 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:05.141258 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:05.141945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.142171 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.142489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] PATH_INFO: `/v3/projects/32f739d9490d4bd485d2a89848a9f36d` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.143605 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] RBAC: Authorizing `identity:delete_project(project_id=32f739d9490d4bd485d2a89848a9f36d)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:05.145737 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:05.191743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32f739d9490d4bd485d2a89848a9f36d', 'request_id': 'req-777b4c2a-1269-4440-b8c4-9a8cb31da45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.197845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32f739d9490d4bd485d2a89848a9f36d', 'request_id': 'req-777b4c2a-1269-4440-b8c4-9a8cb31da45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.205292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32f739d9490d4bd485d2a89848a9f36d', 'request_id': 'req-777b4c2a-1269-4440-b8c4-9a8cb31da45c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.216094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] The token cache is being invalidate because project 32f739d9490d4bd485d2a89848a9f36d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:27:05.216382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.217010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.218519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1375/2739] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:27:05 2022] DELETE /identity/v3/projects/32f739d9490d4bd485d2a89848a9f36d => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:05.222532 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:05.223982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:05.240868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90f8c3d-e274-40aa-a2d5-88140679bd36 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:05.297152 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:05.297911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.298195 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.298500 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] PATH_INFO: `/v3/users/375ad328c80b4646b3a20190675df574` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.299656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=375ad328c80b4646b3a20190675df574)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:05.301583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:05.322535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:05.348615 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '375ad328c80b4646b3a20190675df574'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.356065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '375ad328c80b4646b3a20190675df574'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.363385 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '375ad328c80b4646b3a20190675df574'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.363994 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '375ad328c80b4646b3a20190675df574'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.375671 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4903fd8c-b467-40a4-a7f1-7aa7903952f9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '375ad328c80b4646b3a20190675df574'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.381460 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1365/2740] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:27:05 2022] DELETE /identity/v3/users/375ad328c80b4646b3a20190675df574 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:05.659360 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:05.661391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:05.677373 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-777b4c2a-1269-4440-b8c4-9a8cb31da45c admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:05.733625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:05.742078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:05.742858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.743310 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.743734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] PATH_INFO: `/v3/projects/f8888425c0844e49b92d3b65d1f7591f` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.745044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f8888425c0844e49b92d3b65d1f7591f)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:05.747495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:05.794919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f8888425c0844e49b92d3b65d1f7591f', 'request_id': 'req-bce99218-c9fc-4284-a3db-56f1ce9ac841'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.801176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f8888425c0844e49b92d3b65d1f7591f', 'request_id': 'req-bce99218-c9fc-4284-a3db-56f1ce9ac841'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.809671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f8888425c0844e49b92d3b65d1f7591f', 'request_id': 'req-bce99218-c9fc-4284-a3db-56f1ce9ac841'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.824288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] The token cache is being invalidate because project f8888425c0844e49b92d3b65d1f7591f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:27:05.824594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.825348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-bce99218-c9fc-4284-a3db-56f1ce9ac841 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:27:05.826881 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1376/2741] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:27:05 2022] DELETE /identity/v3/projects/f8888425c0844e49b92d3b65d1f7591f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:05.831975 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.832335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.832679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.853296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:05.854545 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:05.914111 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1366/2742] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:05.922748 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.923033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.923339 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.943865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:05.945075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:05.953349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1377/2743] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:05.957757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:05.968867 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7758a1-3a82-47bf-b34e-49b43230b285 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:05.974145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:05.974854 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:05.975169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:05.975624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:05.976792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:27:05.977341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:05.981219 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:05.994150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1367/2744] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:27:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:27:05.998826 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.008256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1c914-54f2-433c-890a-b40f0f474294 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.013316 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.013918 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.014199 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.014514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.015158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.017031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.046994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1378/2745] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:27:05 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:27:06.051366 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.062819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44530ae1-6538-4cbe-b18b-d6647f0f9357 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.118910 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.119503 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.119843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.120117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.120993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.122930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.144755 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1368/2746] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:27:06 2022] POST /identity/v3/users => generated 290 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:27:06.148923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.160036 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06866077-8242-4d98-bd56-19b00a94c3a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.164564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.165152 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.165431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.165697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.166369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.168064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.177753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1379/2747] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:27:06.181587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.191961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8df38d-3208-424f-8997-cf31ad0b3284 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.196650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.197243 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.197521 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.197811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.198979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.200577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dccdf6-9ead-4fe4-9e31-5f537a9603d4 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.211812 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1369/2748] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:27:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:27:06.216563 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.226557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aec70d4-00f5-47a7-9e86-604b484e40e6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.231477 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.232131 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.232413 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.232682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] PATH_INFO: `/v3/projects/59c345f38d6548cd820affce5406d62c/users/b301d6b1d85b4371abde52727d9b387e/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.247292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=59c345f38d6548cd820affce5406d62c, user_id=b301d6b1d85b4371abde52727d9b387e, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.249928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.267512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1380/2749] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:27:06 2022] PUT /identity/v3/projects/59c345f38d6548cd820affce5406d62c/users/b301d6b1d85b4371abde52727d9b387e/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:06.275145 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90c73-897a-4b31-8c9b-631cebbaec54 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.275706 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90c73-897a-4b31-8c9b-631cebbaec54 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.275985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8b90c73-897a-4b31-8c9b-631cebbaec54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.284365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:06.299381 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-e8b90c73-897a-4b31-8c9b-631cebbaec54 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:06.300587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e8b90c73-897a-4b31-8c9b-631cebbaec54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:06.364181 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1370/2750] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:06.373930 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:06.375780 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:06.434144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb47491b-8208-4226-bd2d-9b83c78c7a7e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:06.437736 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:06.438395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:06.438674 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:06.438938 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:06.448054 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:06.450315 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:06.459733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1381/2751] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:06 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:10.636085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:10.636085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:10.636085 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:10.672010 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] MFA Rules not processed for user `b301d6b1d85b4371abde52727d9b387e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:10.673221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:10.767274 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1371/2752] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:10.772049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:10.783581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6af2a66-208c-49f6-86ed-411f3a8926f1 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:10.789548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:10.790166 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:10.790495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:10.790764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:10.791400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:10.793533 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:10.834710 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1382/2753] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:27:10 2022] POST /identity/v3/projects => generated 379 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:27:10.835888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:10.849259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b237f0-ff78-45ca-a08a-df86ebf58ff6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:10.921834 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:10.922680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:10.923060 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:10.923394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:10.924146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:10.926155 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:10.952188 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1372/2754] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:27:10 2022] POST /identity/v3/users => generated 296 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:27:10.953516 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:10.959327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:10.967289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e2934ae-4c7b-43fb-802e-4adc25c628d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:10.973560 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:10.974321 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:10.974606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:10.974906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:10.975587 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:10.977187 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:10.988807 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1383/2755] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:27:10 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:27:10.992995 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:11.007888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a947b350-d4ae-4160-a7cb-397f929fb515 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:11.013289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:11.013960 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:11.014237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:11.014558 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] PATH_INFO: `/v3/projects/dbd4de50617f4a3585cb620eb58a73c2/users/97bf0511074e409a8b73b615aef5971b/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:11.031804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd4de50617f4a3585cb620eb58a73c2, user_id=97bf0511074e409a8b73b615aef5971b, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:11.034575 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:11.044347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:11.050977 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1373/2756] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:27:10 2022] PUT /identity/v3/projects/dbd4de50617f4a3585cb620eb58a73c2/users/97bf0511074e409a8b73b615aef5971b/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:11.054944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:11.065894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac44a-45a6-4377-9a12-fa31d349568f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:11.128666 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:11.129208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:11.129485 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:11.129754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:11.130430 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:11.132044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b2ffc-b9dc-4c2e-87c7-6bd42eb20d32 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:11.142965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1384/2757] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:27:11 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:27:11.147418 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:11.162569 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62d71a1-e7f0-4cc1-b75a-c7382037c69b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:11.167943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:11.168880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:11.169163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:11.169435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] PATH_INFO: `/v3/projects/dbd4de50617f4a3585cb620eb58a73c2/users/97bf0511074e409a8b73b615aef5971b/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:11.171487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd4de50617f4a3585cb620eb58a73c2, user_id=97bf0511074e409a8b73b615aef5971b, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:11.174143 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:11.185679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1374/2758] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:27:11 2022] PUT /identity/v3/projects/dbd4de50617f4a3585cb620eb58a73c2/users/97bf0511074e409a8b73b615aef5971b/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:27:15.944648 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:15.945181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:15.945181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:15.981896 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] MFA Rules not processed for user `97bf0511074e409a8b73b615aef5971b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:15.983324 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:16.060445 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1385/2759] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:15 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:16.073760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:16.075751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:16.234148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d122d-f0c4-49af-8b03-aafddae7847a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:16.246413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:16.247335 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:16.248042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:16.248470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:16.320179 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:16.321828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:16.332250 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1375/2760] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:16 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 259 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:26.030113 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:26.032650 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:26.119290 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4735e466-db40-4a57-87ff-585e65804d23 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:26.124951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:26.125856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:26.126134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:26.126448 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:26.128644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:26.138086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:26.140899 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:26.143208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd48cd2-0ac5-4b0d-81a7-2fd4cb457c2d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:26.153421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1386/2761] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:26 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:26.229430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:26.239592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df122947-b8ca-4ffc-8cdb-7b3b00ddd5ea service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:26.243280 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:26.243886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:26.244159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:26.244433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:26.246701 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:26.261279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:26.262881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:26.275763 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1376/2762] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:26 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:32.759274 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69176-532a-4111-83e0-237ddac0ec20 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:32.759834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69176-532a-4111-83e0-237ddac0ec20 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:32.759834 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cb69176-532a-4111-83e0-237ddac0ec20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:32.779145 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-3cb69176-532a-4111-83e0-237ddac0ec20 None None] MFA Rules not processed for user `b301d6b1d85b4371abde52727d9b387e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:32.780356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3cb69176-532a-4111-83e0-237ddac0ec20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:32.787369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1387/2763] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:32 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:27:36.270299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=80,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:42.773928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:27:51.836857 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:27:51.846139 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5811ace-3fc0-49dc-98a6-9d6f8616ca91 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:27:51.850340 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:27:51.850937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:51.851210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:51.851480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:51.861421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:27:51.863163 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:27:51.877598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1377/2764] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:27:51 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:27:53.851774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:27:53.852293 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:27:53.852396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:27:53.880284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] MFA Rules not processed for user `545b18260abe47ceb440d0713439c13b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:27:53.882007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:27:53.948609 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1388/2765] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:27:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:28:01.872662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:28:03.936416 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=25 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:28:52.196930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:28:52.206351 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4eec47-62ba-4b95-8a7e-d59330111133 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:28:52.209925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:28:52.210551 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:28:52.210827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:28:52.211093 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:28:52.212786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:28:52.229360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:28:52.231222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:28:52.242661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1378/2766] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:28:52 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:02.237717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:02.692108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:02.694165 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:02.774469 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab4b6be-0ca7-43e8-8df4-b6e95aa6d1e1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:02.779519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:02.780154 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:02.780436 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:02.780709 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] PATH_INFO: `/v3/users/545b18260abe47ceb440d0713439c13b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:02.781855 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] RBAC: Authorizing `identity:delete_user(user_id=545b18260abe47ceb440d0713439c13b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:02.783876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:02.835574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '545b18260abe47ceb440d0713439c13b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:02.851893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '545b18260abe47ceb440d0713439c13b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:02.852606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '545b18260abe47ceb440d0713439c13b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:02.853214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '545b18260abe47ceb440d0713439c13b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:02.861876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d82d0c13-9d97-4131-932e-6e953c70479c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '545b18260abe47ceb440d0713439c13b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:02.874639 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1389/2767] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:02 2022] DELETE /identity/v3/users/545b18260abe47ceb440d0713439c13b => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:03.056892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.076347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001000db-d1eb-4456-9eb6-91a0d1506ebb service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.135949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.136589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.136871 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.137655 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] PATH_INFO: `/v3/projects/9c9311c57fdd4250a16560bd8c2e0ffc` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.139137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c9311c57fdd4250a16560bd8c2e0ffc)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.141056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.191811 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c9311c57fdd4250a16560bd8c2e0ffc', 'request_id': 'req-c1978ce0-2b6f-4412-8584-19dc6add909a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:03.200258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c9311c57fdd4250a16560bd8c2e0ffc', 'request_id': 'req-c1978ce0-2b6f-4412-8584-19dc6add909a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:03.207541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c9311c57fdd4250a16560bd8c2e0ffc', 'request_id': 'req-c1978ce0-2b6f-4412-8584-19dc6add909a'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:03.219996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] The token cache is being invalidate because project 9c9311c57fdd4250a16560bd8c2e0ffc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:03.220284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:03.220996 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-c1978ce0-2b6f-4412-8584-19dc6add909a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:03.222745 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1379/2768] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:03 2022] DELETE /identity/v3/projects/9c9311c57fdd4250a16560bd8c2e0ffc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:03.227497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.227788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.228065 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.371816 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:03.373117 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:03.451860 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1390/2769] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:03.453484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.453924 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.454099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.474658 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:03.475881 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:03.484067 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1380/2770] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:29:03.488403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.499013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e176c-d314-4ee5-bac9-b48a493e6b21 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.503786 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.504451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.504731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.504995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.505636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:03.505935 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.507544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.532730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1391/2771] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:03.533974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.543862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dff751-04c9-4fc4-8469-2b2e3c14d1cb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.548538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.549138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.549421 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.549690 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.550393 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.552345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.592599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1381/2772] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/projects => generated 365 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:29:03.596400 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.606820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bcd771c-a722-416f-a8f7-78efa51f69d4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.612004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:03.669832 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.670466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.670743 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.671008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.671634 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.673875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.697640 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1392/2773] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:29:03.702299 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.716444 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b53349-63e3-4d5d-8f97-9b5084fa3fbd admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.720822 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.721427 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.721702 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.721993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.722661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.724256 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.735773 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1382/2774] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:03.738817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.752214 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea082b5-c59a-4107-9cca-31cf0c394b5f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.757170 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.757779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.758058 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.758349 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.759191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.760779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.773936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1393/2775] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:03 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:03.779990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.790979 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ec883b-0523-4394-ab36-c0904a4c2eae admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.795714 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.796365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.796682 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.796956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] PATH_INFO: `/v3/projects/cc1b38dc808a45f2a343821be3a05109/users/8ddcfde9961a4e54b80dfed5cddb7b8d/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.798370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.800183 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:03.819138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc1b38dc808a45f2a343821be3a05109, user_id=8ddcfde9961a4e54b80dfed5cddb7b8d, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:03.821820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4896ebf-c216-452a-a5da-359232c10fb2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:03.835963 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1383/2776] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:03 2022] PUT /identity/v3/projects/cc1b38dc808a45f2a343821be3a05109/users/8ddcfde9961a4e54b80dfed5cddb7b8d/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:03.848730 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.849109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.849387 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.861263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83fbfaa0-0a52-4630-836f-be9342db1a00 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:03.879715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] MFA Rules not processed for user `8ddcfde9961a4e54b80dfed5cddb7b8d`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:03.880954 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:03.887466 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:03.919528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:03.920220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:03.920500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:03.920768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:03.922597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:03.969925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1384/2777] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:29:03.970895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:03.977496 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:04.007061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.011217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.034284 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1394/2778] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:03 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:29:04.063311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0949c1-b5b7-4adc-9980-f4b1ad898450 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.070591 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.071156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.071433 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.071735 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.072333 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.074982 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.110792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1385/2779] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:03 2022] POST /identity/v3/projects => generated 365 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:04.113142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.124263 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71db10fd-b99a-488b-ad0c-199887d25ff6 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.183233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.183877 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.184731 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.185144 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.185995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.188086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.212727 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1395/2780] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:04 2022] POST /identity/v3/users => generated 289 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:29:04.217436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.228012 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4a74c-26b6-4747-b99a-c6a3a09ef29d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.233463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.234131 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.234492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.234823 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.235780 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.237455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.248584 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1386/2781] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:04.252894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.262936 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46981a4a-a326-42dd-b991-8a409b641da9 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.267682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.268358 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.268638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.268906 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] PATH_INFO: `/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44/users/c6cf18d83e7948eb89b9c90f3f46a3ed/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.286142 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8baa5eb0d2944ecbab51ecb7a1ffa44, user_id=c6cf18d83e7948eb89b9c90f3f46a3ed, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.289185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.302817 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1396/2782] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:04 2022] PUT /identity/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44/users/c6cf18d83e7948eb89b9c90f3f46a3ed/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:04.306753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.319128 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60be9b46-4848-4fb9-94f8-6ed7869b4025 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.381847 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.382501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.382771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.383046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.383688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.385282 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-036749aa-78dd-4cc2-b569-a79b1854f989 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.396720 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1387/2783] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:04.400793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.412384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579a047-18d3-44b1-8e96-929828c57128 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.417866 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.419239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.419524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.419791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] PATH_INFO: `/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44/users/c6cf18d83e7948eb89b9c90f3f46a3ed/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.421726 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8baa5eb0d2944ecbab51ecb7a1ffa44, user_id=c6cf18d83e7948eb89b9c90f3f46a3ed, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.424522 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.437704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1397/2784] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:04 2022] PUT /identity/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44/users/c6cf18d83e7948eb89b9c90f3f46a3ed/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:04.445901 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.446189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.446563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.483370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] MFA Rules not processed for user `c6cf18d83e7948eb89b9c90f3f46a3ed`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:04.484665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:04.490439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:04.567806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1388/2785] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:04 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:04.581248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:04.583177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:04.667564 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8233ec8a-b7e7-4961-93bb-39357e4164e2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:04.671806 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:04.672410 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:04.672682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:04.673053 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:04.682010 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:04.683444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b44deb6-c22d-4776-80fd-b13624aaf330 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:04.692970 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1398/2786] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:04 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:05.153178 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:05.218899 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed3ef3-ec4e-4a2e-ba93-52b439712238 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:05.224186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:05.224792 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:05.225068 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:05.225341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] PATH_INFO: `/v3/users/8ddcfde9961a4e54b80dfed5cddb7b8d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:05.226585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ddcfde9961a4e54b80dfed5cddb7b8d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:05.228640 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:05.285806 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ddcfde9961a4e54b80dfed5cddb7b8d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.299169 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ddcfde9961a4e54b80dfed5cddb7b8d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.309127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ddcfde9961a4e54b80dfed5cddb7b8d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.310109 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ddcfde9961a4e54b80dfed5cddb7b8d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.328302 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ddcfde9961a4e54b80dfed5cddb7b8d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.334231 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1389/2787] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:05 2022] DELETE /identity/v3/users/8ddcfde9961a4e54b80dfed5cddb7b8d => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:05.339005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:05.339506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:05.339594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:05.362195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:05.363443 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:05.377879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:05.444130 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1399/2788] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:05 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:05.451632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:05.453074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:05.483630 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:05.518907 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19873a2e-577c-41af-8184-8df0e350f0bc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:05.522997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:05.523751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:05.524035 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:05.524306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:05.534182 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:05.535699 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:05.546897 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1390/2789] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:05 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:05.861928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:05.863393 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:05.877366 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c738aa6-7471-49d3-b353-610efb52c160 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:05.883194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:05.884231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:05.884540 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:05.884870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] PATH_INFO: `/v3/projects/cc1b38dc808a45f2a343821be3a05109` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:05.886176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc1b38dc808a45f2a343821be3a05109)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:05.888681 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:05.938820 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc1b38dc808a45f2a343821be3a05109', 'request_id': 'req-4af6109f-a9b4-404b-81c2-4385e5f051dd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.945185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc1b38dc808a45f2a343821be3a05109', 'request_id': 'req-4af6109f-a9b4-404b-81c2-4385e5f051dd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.951976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc1b38dc808a45f2a343821be3a05109', 'request_id': 'req-4af6109f-a9b4-404b-81c2-4385e5f051dd'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.961912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] The token cache is being invalidate because project cc1b38dc808a45f2a343821be3a05109 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:05.962201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.962854 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:05.964330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1400/2790] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:05 2022] DELETE /identity/v3/projects/cc1b38dc808a45f2a343821be3a05109 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:05.969018 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:05.979536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61af00f1-0496-4d01-adee-e75f128180ac service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.037666 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.038322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.038717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.038917 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] PATH_INFO: `/v3/users/c6cf18d83e7948eb89b9c90f3f46a3ed` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.040091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6cf18d83e7948eb89b9c90f3f46a3ed)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.042001 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.091092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6cf18d83e7948eb89b9c90f3f46a3ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.098801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6cf18d83e7948eb89b9c90f3f46a3ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.106581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6cf18d83e7948eb89b9c90f3f46a3ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.107390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6cf18d83e7948eb89b9c90f3f46a3ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.119679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-4594b3b4-507d-4723-be03-d627d773d8c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6cf18d83e7948eb89b9c90f3f46a3ed'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.126920 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1391/2791] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:05 2022] DELETE /identity/v3/users/c6cf18d83e7948eb89b9c90f3f46a3ed => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:06.418457 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.419945 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:06.433114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af6109f-a9b4-404b-81c2-4385e5f051dd admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.493440 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.494068 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.494383 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.494658 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] PATH_INFO: `/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.495821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8baa5eb0d2944ecbab51ecb7a1ffa44)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.498271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.540150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:06.543597 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8baa5eb0d2944ecbab51ecb7a1ffa44', 'request_id': 'req-36474715-a82f-49f6-8606-02a2151b8bc3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.551514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8baa5eb0d2944ecbab51ecb7a1ffa44', 'request_id': 'req-36474715-a82f-49f6-8606-02a2151b8bc3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.560546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8baa5eb0d2944ecbab51ecb7a1ffa44', 'request_id': 'req-36474715-a82f-49f6-8606-02a2151b8bc3'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.571994 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] The token cache is being invalidate because project c8baa5eb0d2944ecbab51ecb7a1ffa44 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:06.572282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.572933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-36474715-a82f-49f6-8606-02a2151b8bc3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:06.574451 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1401/2792] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:06 2022] DELETE /identity/v3/projects/c8baa5eb0d2944ecbab51ecb7a1ffa44 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:06.580076 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.580572 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.580681 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.601013 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:06.602306 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:06.654425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:06.675177 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1392/2793] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:06.682875 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.683159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.683423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.704680 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:06.705947 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:06.714225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1402/2794] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:06 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:06.718737 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.728398 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd0e496-2e8b-4bf4-bbad-2afabb95863e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.733304 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.733909 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.734186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.734495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.735144 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:06.735445 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.737046 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.749294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1393/2795] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:06.753835 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.764512 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f452a2-bb56-472a-be25-4a9bf58a95e1 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.770047 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.770690 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.770969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.771238 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.771887 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.773802 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.810984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1403/2796] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:06 2022] POST /identity/v3/projects => generated 381 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:06.815669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.831042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef48b4b-9992-4bf5-9abd-7c7383169a5b admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.893856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.894502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.895044 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.895320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.895949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.897877 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.924052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1394/2797] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:06 2022] POST /identity/v3/users => generated 291 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:06.925509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.936160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4e6758-a0c4-4034-b3ea-5af8b2babf3f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.941232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.941856 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.942134 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.942437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.943180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.944895 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.956444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1404/2798] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:06.960644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:06.971939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de0c1a3-e46a-480d-bde1-f10b4cd17d49 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:06.976850 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:06.977470 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:06.977748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:06.978021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:06.978722 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:06.980349 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421fb8c0-f580-458f-b923-bbd4ad7902e2 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:06.996081 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1395/2799] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:06.997845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.008185 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e818af25-e273-44fe-9c0d-5b1b9a70dd9b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.013548 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.014273 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.014577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.014850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] PATH_INFO: `/v3/projects/785d1dfb05014973ac35f65fd1c1676d/users/48fae56fff4b40c1a5d3248b7eea3dee/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.032588 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] RBAC: Authorizing `identity:create_grant(project_id=785d1dfb05014973ac35f65fd1c1676d, user_id=48fae56fff4b40c1a5d3248b7eea3dee, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.035412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.051179 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1405/2800] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:06 2022] PUT /identity/v3/projects/785d1dfb05014973ac35f65fd1c1676d/users/48fae56fff4b40c1a5d3248b7eea3dee/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:29:07.064094 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.064522 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.064801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.094986 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] MFA Rules not processed for user `48fae56fff4b40c1a5d3248b7eea3dee`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:07.096056 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:07.166100 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1396/2801] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:07.171195 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.234821 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebd2fb-6322-407a-aae5-0a569ebb0935 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.239380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.239998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.240257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.240527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.241155 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.243070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.272098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1406/2802] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:07 2022] POST /identity/v3/projects => generated 379 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:07.276378 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.289538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61e58c5-9cff-437a-bb16-245aabf6c026 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.328537 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:07.347987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.348592 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.348865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.349134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.349768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.351715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.377456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1397/2803] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:07 2022] POST /identity/v3/users => generated 296 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:07.381465 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.392265 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd11317-61dc-4a7f-a78e-2032cb1ab9c0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.397541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.398160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.398535 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.398811 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.399461 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.401062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.412670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1407/2804] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:07.416787 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.428828 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3026b1-a747-46d0-8051-ef8b437ff84f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.433749 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.434495 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.434796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.435563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] PATH_INFO: `/v3/projects/b0041fc77f414dba91f8bb11672952c0/users/24d4320cae5341e9b597ea9fb57974aa/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.452203 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0041fc77f414dba91f8bb11672952c0, user_id=24d4320cae5341e9b597ea9fb57974aa, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.454896 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.469048 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1398/2805] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:07 2022] PUT /identity/v3/projects/b0041fc77f414dba91f8bb11672952c0/users/24d4320cae5341e9b597ea9fb57974aa/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:07.473033 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.483771 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3641bd31-8e12-4a7f-8e65-eb5b98d13e0b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.524286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:07.544539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.545334 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.545610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.545876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.546569 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.548147 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41b88d5-7324-4a16-85be-a968c654c848 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.558063 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1408/2806] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:07 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:07.562703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.573112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e0bd95-556d-449d-9be1-8399a3c5749e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.578843 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.579767 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.580193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.581159 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] PATH_INFO: `/v3/projects/b0041fc77f414dba91f8bb11672952c0/users/24d4320cae5341e9b597ea9fb57974aa/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.584618 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0041fc77f414dba91f8bb11672952c0, user_id=24d4320cae5341e9b597ea9fb57974aa, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.587585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.606301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1399/2807] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:07 2022] PUT /identity/v3/projects/b0041fc77f414dba91f8bb11672952c0/users/24d4320cae5341e9b597ea9fb57974aa/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:07.616850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.617222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.617568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.649057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] MFA Rules not processed for user `24d4320cae5341e9b597ea9fb57974aa`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:07.650308 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:07.722409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1409/2808] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:07.735090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:07.736817 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:07.797146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d7ea62-735e-4567-96df-ae9e2596dacc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:07.801886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:07.802581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:07.802865 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:07.803134 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:07.814161 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:07.815953 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:07.827141 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1400/2809] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:08.191460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:08.202964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845a7e8d-41e1-4e9f-8ead-3eab930fe31c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:08.211350 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d4320cae5341e9b597ea9fb57974aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0041fc77f414dba91f8bb11672952c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:08.211982 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:08.212272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:08.212574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:08.213235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:08.213539 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:08.215774 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:08.228519 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1410/2810] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:08.233383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:08.242838 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ed6d9e4-cf38-477a-8197-58308df84b41 service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:08.247939 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d4320cae5341e9b597ea9fb57974aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0041fc77f414dba91f8bb11672952c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:08.248539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:08.248816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:08.249084 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:08.249710 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:08.251644 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:08.280557 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1401/2811] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:08 2022] POST /identity/v3/projects => generated 355 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:08.285327 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:08.298336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2cea3a-9b76-4a3d-a84d-10a5f7b92e30 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:08.366356 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d4320cae5341e9b597ea9fb57974aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0041fc77f414dba91f8bb11672952c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:08.366984 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:08.367289 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:08.367536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:08.368340 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:08.368644 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:08.370233 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:08.382822 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1411/2812] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:08.421804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:08.432578 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cc41dc-fa72-4465-995e-f9220192dcfc tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:08.438113 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '24d4320cae5341e9b597ea9fb57974aa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b0041fc77f414dba91f8bb11672952c0', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:08.438935 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:08.439218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:08.439519 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] PATH_INFO: `/v3/projects/94f29990bc3f47b7ba15300774edfccb` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:08.440713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorizing `identity:delete_project(project_id=94f29990bc3f47b7ba15300774edfccb)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:08.442916 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:08.494866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94f29990bc3f47b7ba15300774edfccb', 'request_id': 'req-86e0c5a2-4c12-4b84-88da-33df5dee77f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.504820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94f29990bc3f47b7ba15300774edfccb', 'request_id': 'req-86e0c5a2-4c12-4b84-88da-33df5dee77f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.511845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94f29990bc3f47b7ba15300774edfccb', 'request_id': 'req-86e0c5a2-4c12-4b84-88da-33df5dee77f0'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.516527 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:INSERT=7,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:08.524305 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] The token cache is being invalidate because project 94f29990bc3f47b7ba15300774edfccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:08.525042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.525777 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-86e0c5a2-4c12-4b84-88da-33df5dee77f0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.527752 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1402/2813] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:08 2022] DELETE /identity/v3/projects/94f29990bc3f47b7ba15300774edfccb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:08.742925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:08.744750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:08.799296 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:08.832635 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd123fb-ed19-43d4-a220-688c261a1fc0 tempest-VolumeQuotaClassesTest-179277070 tempest-VolumeQuotaClassesTest-179277070-project-admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:08.904561 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:08.905480 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:08.905960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:08.906431 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] PATH_INFO: `/v3/users/48fae56fff4b40c1a5d3248b7eea3dee` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:08.908098 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48fae56fff4b40c1a5d3248b7eea3dee)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:08.910602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:08.971040 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48fae56fff4b40c1a5d3248b7eea3dee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.983423 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48fae56fff4b40c1a5d3248b7eea3dee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.984239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48fae56fff4b40c1a5d3248b7eea3dee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.984893 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48fae56fff4b40c1a5d3248b7eea3dee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:08.992946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48fae56fff4b40c1a5d3248b7eea3dee'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.009079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1412/2814] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:08 2022] DELETE /identity/v3/users/48fae56fff4b40c1a5d3248b7eea3dee => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:09.014727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:09.015246 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:09.015336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:09.036040 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:09.037294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:09.108509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1403/2815] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:09 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:09.118361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:09.119964 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:09.190581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a885ee-ba1d-4552-8b49-ff7736671da3 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:09.194222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:09.194859 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:09.195137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:09.195409 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:09.205307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:09.206766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:09.216803 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1413/2816] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:09 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:09.636695 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:09.638390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:09.651189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa10af27-5d4a-420f-9f8b-dc009e0c72c8 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:09.657228 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:09.658328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:09.658610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:09.658908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] PATH_INFO: `/v3/projects/785d1dfb05014973ac35f65fd1c1676d` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:09.660033 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=785d1dfb05014973ac35f65fd1c1676d)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:09.661930 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:09.729176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '785d1dfb05014973ac35f65fd1c1676d', 'request_id': 'req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.739713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '785d1dfb05014973ac35f65fd1c1676d', 'request_id': 'req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.755258 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '785d1dfb05014973ac35f65fd1c1676d', 'request_id': 'req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.769993 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] The token cache is being invalidate because project 785d1dfb05014973ac35f65fd1c1676d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:09.770457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.771117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.774292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1404/2817] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:09 2022] DELETE /identity/v3/projects/785d1dfb05014973ac35f65fd1c1676d => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:29:09.776976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:09.788135 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2157c21-506d-48e6-938d-71e019310843 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:09.797876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:09.867028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:09.868590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:09.868590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:09.868590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] PATH_INFO: `/v3/users/24d4320cae5341e9b597ea9fb57974aa` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:09.869489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=24d4320cae5341e9b597ea9fb57974aa)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:09.871486 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:09.942653 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24d4320cae5341e9b597ea9fb57974aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.960015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24d4320cae5341e9b597ea9fb57974aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.961041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24d4320cae5341e9b597ea9fb57974aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.962387 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24d4320cae5341e9b597ea9fb57974aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.973037 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-d73866af-167d-43af-849b-512cfd1225ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24d4320cae5341e9b597ea9fb57974aa'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:09.991016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1414/2818] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:09 2022] DELETE /identity/v3/users/24d4320cae5341e9b597ea9fb57974aa => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:10.515454 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:10.516998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:10.532238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb184dff-ef6d-43ea-abc0-b60a38eeec8c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:10.598636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:10.599570 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.600023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.600291 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] PATH_INFO: `/v3/projects/b0041fc77f414dba91f8bb11672952c0` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.601408 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0041fc77f414dba91f8bb11672952c0)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:10.603330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:10.632785 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:10.658451 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0041fc77f414dba91f8bb11672952c0', 'request_id': 'req-a3ea4764-92ea-4a83-9b94-64d06715b977'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:10.667392 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0041fc77f414dba91f8bb11672952c0', 'request_id': 'req-a3ea4764-92ea-4a83-9b94-64d06715b977'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:10.674269 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0041fc77f414dba91f8bb11672952c0', 'request_id': 'req-a3ea4764-92ea-4a83-9b94-64d06715b977'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:10.685456 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] The token cache is being invalidate because project b0041fc77f414dba91f8bb11672952c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:10.685739 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:10.686469 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a3ea4764-92ea-4a83-9b94-64d06715b977 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:10.688119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1405/2819] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:10 2022] DELETE /identity/v3/projects/b0041fc77f414dba91f8bb11672952c0 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:10.693066 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.693420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.693686 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.713646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:10.714876 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:10.781088 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1415/2820] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:10.789248 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.789677 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.789961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.812638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:10.813626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:10.822613 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1406/2821] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:10 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:10.827385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:10.837583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0c4af1-7c95-403b-9939-89603f3371dc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:10.842839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:10.843662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.843995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.844259 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.844928 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:10.845224 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:10.846865 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:10.860920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1416/2822] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Aug 30 04:29:10.865196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:10.875674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdbb129-0be2-4d73-8348-289440fb475a None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:10.880759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:10.881370 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.881645 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.881912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.882589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:10.884508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:10.915354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1407/2823] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:10 2022] POST /identity/v3/projects => generated 409 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:10.919833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:10.930625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bb1e34-3a9d-42f5-a05b-fef85ee7e1b2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:10.949108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:10.993939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:10.994586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:10.995208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:10.995208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:10.995836 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:10.998362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.020762 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1417/2824] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:10 2022] POST /identity/v3/users => generated 305 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:11.024968 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.035933 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a91fcb3-7c9a-4d32-ba64-0dad9fa1feb0 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.040626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.041222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.041501 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.041774 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.042450 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.045313 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.057320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1408/2825] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:11.061675 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.072791 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f972e0ab-6799-439c-a611-8bf1efd00ae7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.080338 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.080953 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.081227 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.081497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.082149 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.085629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93aa5a14-3edf-4ebc-b03f-e43d8ab266e4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.098302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1418/2826] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:11.102562 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.113723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a06804-c7c1-4520-ae19-146875cb6648 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.118413 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.119063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.119337 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.119608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] PATH_INFO: `/v3/projects/c36f3d7e83ea43ebb340dfc5f1805579/users/06f55f37acb744469303e2676c81765b/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.134339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c36f3d7e83ea43ebb340dfc5f1805579, user_id=06f55f37acb744469303e2676c81765b, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.137022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.153624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1409/2827] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:11 2022] PUT /identity/v3/projects/c36f3d7e83ea43ebb340dfc5f1805579/users/06f55f37acb744469303e2676c81765b/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:11.222795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.223235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.223581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.258276 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] MFA Rules not processed for user `06f55f37acb744469303e2676c81765b`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:11.259520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:11.328159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1419/2828] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:11 2022] POST /identity/v3/auth/tokens => generated 3286 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:11.332599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.397944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceec7af3-7734-453a-aa7b-d3213450b425 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.402803 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.404197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.404479 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.404753 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.405583 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.408489 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.436667 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1410/2829] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:11 2022] POST /identity/v3/projects => generated 409 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:29:11.440946 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.452965 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d760b-5347-49eb-9362-586a22bf1411 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.519292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.520219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.520631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.521024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.521642 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.523544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.545662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1420/2830] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:11 2022] POST /identity/v3/users => generated 311 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:11.550493 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.564360 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d095bde-01af-4ba4-80ae-f6d2498ad676 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.569712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.570835 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.571119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.571389 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.572043 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.573669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.590039 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1411/2831] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:11.591929 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.604502 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16180214-417a-48f7-a203-8e45d9850105 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.613472 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.614355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.614700 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.615034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] PATH_INFO: `/v3/projects/ddb2f46e021c4f9484ddfe123b885a25/users/9434e3769b0f485d82499a9b0d52282c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.633861 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb2f46e021c4f9484ddfe123b885a25, user_id=9434e3769b0f485d82499a9b0d52282c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.636615 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.641880 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:11.655013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1421/2832] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:11 2022] PUT /identity/v3/projects/ddb2f46e021c4f9484ddfe123b885a25/users/9434e3769b0f485d82499a9b0d52282c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:11.658844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.670201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f32998-73dc-4d41-aab9-2fb3994ef883 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.692347 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:11.755911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.755911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.755911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.755911 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.757345 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.759208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b334c783-ca43-46f8-827b-b5c345aa85d7 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.771293 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1412/2833] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:11 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:11.775402 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.787201 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be23971b-a5d0-438b-a912-4ee3a714d489 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:11.791943 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:11.792598 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.792879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.793217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] PATH_INFO: `/v3/projects/ddb2f46e021c4f9484ddfe123b885a25/users/9434e3769b0f485d82499a9b0d52282c/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.795979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddb2f46e021c4f9484ddfe123b885a25, user_id=9434e3769b0f485d82499a9b0d52282c, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:11.798646 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:11.811768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1422/2834] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:11 2022] PUT /identity/v3/projects/ddb2f46e021c4f9484ddfe123b885a25/users/9434e3769b0f485d82499a9b0d52282c/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:11.822051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:11.822364 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:11.822636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:11.853680 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] MFA Rules not processed for user `9434e3769b0f485d82499a9b0d52282c`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:11.855103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:11.930621 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1413/2835] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:11 2022] POST /identity/v3/auth/tokens => generated 3353 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:29:11.939437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:11.941151 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:12.006363 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ed4c9b-faf8-4a1e-8773-d99c29aa50c4 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:12.010112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:12.010750 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:12.011025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:12.011292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:12.023509 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:12.025025 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:12.036255 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1423/2836] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:11 2022] GET /identity/v3/auth/tokens => generated 3353 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:12.082241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:12.084207 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:12.156023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfcd606-8e09-4734-b4c1-5832e5b81831 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:12.160453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:12.161238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:12.161638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:12.161998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:12.163956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:12.204212 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:12.212837 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbdb58f1-e3e0-4399-af21-f57e4a5b7378 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:12.216927 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:12.217528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:12.217870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:12.218076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:12.226533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:12.241425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:12.243638 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:12.260132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1414/2837] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:12.286203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:12.287934 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:12.300246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1424/2838] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:12 2022] GET /identity/v3/auth/tokens => generated 3286 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:13.909895 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:13.911999 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:13.994203 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:13.996018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:14.023799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bceee2d0-0f7b-46f5-9009-d15c41696141 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:14.027365 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:14.028651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:14.029015 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:14.029405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] PATH_INFO: `/v3/users/7ba21f12574c4b9281d090651174788a` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:14.030819 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ba21f12574c4b9281d090651174788a)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:14.032972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:14.077049 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3520339-8d9a-49be-a4c4-4aa0837b3318 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:14.084497 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:14.085117 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:14.085463 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:14.085688 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:14.087574 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:14.092487 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ba21f12574c4b9281d090651174788a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.100535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:14.102499 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:14.115880 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1415/2839] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:13 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:14.117248 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ba21f12574c4b9281d090651174788a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.118075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ba21f12574c4b9281d090651174788a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.118742 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ba21f12574c4b9281d090651174788a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.130150 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ba21f12574c4b9281d090651174788a'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.135885 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:INSERT=3,keystone:SELECT=87,keystone:DELETE=10 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:14.158429 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1425/2840] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:13 2022] DELETE /identity/v3/users/7ba21f12574c4b9281d090651174788a => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:14.332346 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:14.344717 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc98c13-17e3-4061-91f6-a9e4b6fd73e2 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:14.408363 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:14.408923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:14.409201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:14.409471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] PATH_INFO: `/v3/projects/b0f04feca7a349e5a656f166dae05354` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:14.410709 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f04feca7a349e5a656f166dae05354)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:14.412632 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:14.472771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f04feca7a349e5a656f166dae05354', 'request_id': 'req-56bfe5eb-3e19-48f0-830c-53ef33d4becc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.480061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f04feca7a349e5a656f166dae05354', 'request_id': 'req-56bfe5eb-3e19-48f0-830c-53ef33d4becc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.487197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f04feca7a349e5a656f166dae05354', 'request_id': 'req-56bfe5eb-3e19-48f0-830c-53ef33d4becc'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.498153 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] The token cache is being invalidate because project b0f04feca7a349e5a656f166dae05354 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:14.498481 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.499138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.500636 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1416/2841] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:14 2022] DELETE /identity/v3/projects/b0f04feca7a349e5a656f166dae05354 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:14.504618 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:14.506021 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:14.519544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66c6d1a8-618c-44ab-bb6a-7698b956e16f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:14.577378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:14.577998 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:14.578297 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:14.578568 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] PATH_INFO: `/v3/users/6cbc4765c42142a1aad99d6cc830e1b0` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:14.579684 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cbc4765c42142a1aad99d6cc830e1b0)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:14.581582 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:14.634271 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cbc4765c42142a1aad99d6cc830e1b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.647691 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cbc4765c42142a1aad99d6cc830e1b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.648403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cbc4765c42142a1aad99d6cc830e1b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.649007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cbc4765c42142a1aad99d6cc830e1b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.656668 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cbc4765c42142a1aad99d6cc830e1b0'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.668920 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1426/2842] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:14 2022] DELETE /identity/v3/users/6cbc4765c42142a1aad99d6cc830e1b0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:14.847964 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:14.849453 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:14.863344 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56bfe5eb-3e19-48f0-830c-53ef33d4becc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:14.872589 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:14.925316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:14.925961 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:14.926132 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:14.926441 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] PATH_INFO: `/v3/projects/9a7263aaa79b4900afd0bedd9969ece9` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:14.927610 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7263aaa79b4900afd0bedd9969ece9)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:14.929523 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:14.977054 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7263aaa79b4900afd0bedd9969ece9', 'request_id': 'req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.984331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7263aaa79b4900afd0bedd9969ece9', 'request_id': 'req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:14.991058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7263aaa79b4900afd0bedd9969ece9', 'request_id': 'req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:15.002793 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] The token cache is being invalidate because project 9a7263aaa79b4900afd0bedd9969ece9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:15.003074 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:15.003772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0ae5e983-c6fb-4714-bed7-56aaae9e94ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:15.005442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1417/2843] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:14 2022] DELETE /identity/v3/projects/9a7263aaa79b4900afd0bedd9969ece9 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:19.778300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:19.779894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:19.792871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da96725b-02e3-4c5e-bd37-f343bddb3f1a admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:19.853773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:19.854454 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:19.854733 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:19.855352 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] PATH_INFO: `/v3/users/06f55f37acb744469303e2676c81765b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:19.857405 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] RBAC: Authorizing `identity:delete_user(user_id=06f55f37acb744469303e2676c81765b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:19.859344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:19.873845 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:19.905528 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06f55f37acb744469303e2676c81765b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:19.917163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06f55f37acb744469303e2676c81765b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:19.917813 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06f55f37acb744469303e2676c81765b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:19.918976 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06f55f37acb744469303e2676c81765b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:19.927344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06f55f37acb744469303e2676c81765b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:19.940176 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1427/2844] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:19 2022] DELETE /identity/v3/users/06f55f37acb744469303e2676c81765b => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:19.944816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f561-e436-457f-8607-6878c0288443 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:19.945317 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f561-e436-457f-8607-6878c0288443 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:19.945425 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f206f561-e436-457f-8607-6878c0288443 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:19.964581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-f206f561-e436-457f-8607-6878c0288443 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:19.966614 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f206f561-e436-457f-8607-6878c0288443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:20.028998 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1418/2845] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:19 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:20.038830 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:20.040257 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:20.096805 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f6b900-fdbf-4939-8161-b6869c74379b admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:20.100514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:20.101114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:20.101394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:20.101664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:20.111254 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:20.112651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:20.122302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1428/2846] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:20 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:20.392021 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206f561-e436-457f-8607-6878c0288443 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:20.393845 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-f206f561-e436-457f-8607-6878c0288443 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:20.409311 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f206f561-e436-457f-8607-6878c0288443 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:20.413978 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:20.414643 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:20.414921 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:20.415191 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] PATH_INFO: `/v3/projects/c36f3d7e83ea43ebb340dfc5f1805579` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:20.416314 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c36f3d7e83ea43ebb340dfc5f1805579)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:20.418289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:20.458397 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:20.468661 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c36f3d7e83ea43ebb340dfc5f1805579', 'request_id': 'req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.476069 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c36f3d7e83ea43ebb340dfc5f1805579', 'request_id': 'req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.483123 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c36f3d7e83ea43ebb340dfc5f1805579', 'request_id': 'req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.493756 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] The token cache is being invalidate because project c36f3d7e83ea43ebb340dfc5f1805579 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:20.494045 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.494727 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.496253 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1419/2847] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:20 2022] DELETE /identity/v3/projects/c36f3d7e83ea43ebb340dfc5f1805579 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:20.501026 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:20.510939 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba796df-18c3-4b86-ac18-4ece71325869 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:20.567793 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:20.568660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:20.568962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:20.569220 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] PATH_INFO: `/v3/users/9434e3769b0f485d82499a9b0d52282c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:20.570401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9434e3769b0f485d82499a9b0d52282c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:20.572288 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:20.609534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:20.619091 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9434e3769b0f485d82499a9b0d52282c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.631468 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9434e3769b0f485d82499a9b0d52282c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.632160 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9434e3769b0f485d82499a9b0d52282c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.632770 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9434e3769b0f485d82499a9b0d52282c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.640229 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-56df0fce-470b-4d35-9b47-fdc2daf23487 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9434e3769b0f485d82499a9b0d52282c'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:20.652506 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1429/2848] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:20 2022] DELETE /identity/v3/users/9434e3769b0f485d82499a9b0d52282c => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:20.921858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:20.923395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:20.937937 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdb5f4a-f34f-4690-9bf6-c38bfc4d5266 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:20.996175 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:20.997083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:20.997083 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:20.997449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] PATH_INFO: `/v3/projects/ddb2f46e021c4f9484ddfe123b885a25` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:20.998597 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddb2f46e021c4f9484ddfe123b885a25)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.000498 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.049639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddb2f46e021c4f9484ddfe123b885a25', 'request_id': 'req-0d365709-6210-4128-9a51-b246fe38c2f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:21.057088 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddb2f46e021c4f9484ddfe123b885a25', 'request_id': 'req-0d365709-6210-4128-9a51-b246fe38c2f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:21.063565 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddb2f46e021c4f9484ddfe123b885a25', 'request_id': 'req-0d365709-6210-4128-9a51-b246fe38c2f8'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:21.073859 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] The token cache is being invalidate because project ddb2f46e021c4f9484ddfe123b885a25 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:21.074148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:21.075448 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-0d365709-6210-4128-9a51-b246fe38c2f8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:21.077103 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1420/2849] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:20 2022] DELETE /identity/v3/projects/ddb2f46e021c4f9484ddfe123b885a25 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:21.081958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.082471 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.082580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.102790 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:21.104003 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:21.163746 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1430/2850] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:21.173125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.173339 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.173608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.194328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:21.195533 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:21.204020 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1421/2851] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:21.208239 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.218717 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7fa0009-5618-4a2c-a981-de18ff72eea4 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.223426 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.224016 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.224294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.224593 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.225234 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81502) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:21.225536 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.227159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.238814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1431/2852] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:21.243032 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.252477 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340bea4e-2e2b-43b4-ac2d-51acf0779c18 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.257148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.257760 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.258037 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.258342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.258970 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.260856 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.289338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1422/2853] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:21.293553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.303679 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be96a567-c2fd-4b13-bcf2-2cf424d963d6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.358523 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.359124 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.359404 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.359672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.360586 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.362499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.384069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1432/2854] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/users => generated 288 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:21.388900 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.399507 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b66627c-6d41-47e7-beee-0bb3eb193d69 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.404832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.405429 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.405713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.405985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.406665 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.408270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.418879 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1423/2855] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:21.422949 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.433090 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd371cae-244e-4d45-b4f9-a179998d5a5d admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.438028 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.438716 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.438997 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.439269 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.439908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.441483 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f363a6-c592-4a42-89a6-33171bd43b7f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.453181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1433/2856] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:21.457538 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.468164 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f718dde1-fee1-4001-be70-0e252de0ee7c admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.473078 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.473715 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.473988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.474279 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] PATH_INFO: `/v3/projects/664c60dcf0b849bf86bde20095ab1ff3/users/5fd372da71c643efbca21f7d18ffc4ac/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.490221 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:21.493669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] RBAC: Authorizing `identity:create_grant(project_id=664c60dcf0b849bf86bde20095ab1ff3, user_id=5fd372da71c643efbca21f7d18ffc4ac, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.497563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.510885 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1424/2857] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:21 2022] PUT /identity/v3/projects/664c60dcf0b849bf86bde20095ab1ff3/users/5fd372da71c643efbca21f7d18ffc4ac/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:29:21.522084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.522396 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.522665 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.549772 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] MFA Rules not processed for user `5fd372da71c643efbca21f7d18ffc4ac`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:21.551031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:21.559580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:21.616108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1434/2858] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:21.620581 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.682997 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2dfa01a-6a33-4a42-a97a-74bf9f0e558d admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.688619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.689160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.689439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.689711 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.690362 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.692260 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.723171 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1425/2859] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:21.727123 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.737898 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c60da0-8d88-44c8-acee-b523695fa07c None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.796833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.797437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.798571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.798571 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.798827 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.800544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.824384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1435/2860] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/users => generated 289 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Aug 30 04:29:21.829430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.839973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d18485a-1685-401f-8f70-3e45e2f2b4fc admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.844662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.845251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.845529 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.845796 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.846464 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.848064 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.859108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1426/2861] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:21.863085 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.873649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b3940be-c0e0-49f4-8ea9-c9012b4c82c6 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.878437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.879031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.879303 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.879567 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.880196 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.881760 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b974f154-9e1f-46c1-93ca-62628579a390 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.891660 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1436/2862] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:21 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:21.895948 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:21.906459 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b11a2c3-3180-4e20-a277-f1cb3a3e2f76 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:21.911268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:21.911908 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] REQUEST_METHOD: `PUT` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.912186 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.912457 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] PATH_INFO: `/v3/projects/9376a9b34bbb46088035a49b97f20c40/users/ce411d75aa164ab28ec6bc05be6bbd59/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.928336 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9376a9b34bbb46088035a49b97f20c40, user_id=ce411d75aa164ab28ec6bc05be6bbd59, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:21.931106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:21.944331 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1427/2863] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:21 2022] PUT /identity/v3/projects/9376a9b34bbb46088035a49b97f20c40/users/ce411d75aa164ab28ec6bc05be6bbd59/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:21.954578 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:21.954870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:21.955146 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:21.985756 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] MFA Rules not processed for user `ce411d75aa164ab28ec6bc05be6bbd59`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:21.987019 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:22.051858 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1437/2864] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:22.056467 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.123266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da34138-7ec7-4436-9c49-ff0a47d96207 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.127918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.128516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.128791 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.129063 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] PATH_INFO: `/v3/projects` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.129683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.131601 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.157784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1428/2865] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:22 2022] POST /identity/v3/projects => generated 375 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:22.162277 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.173851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6593168-93d9-4dd7-8803-bed6caa1fb17 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.187000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:22.231950 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.232562 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.232839 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.233105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] PATH_INFO: `/v3/users` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.233753 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.235693 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.256526 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1438/2866] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:22 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:22.260586 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.271512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-040808ab-5335-424e-95ef-ddd008eaf61f admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.276150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.276757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.277022 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.277296 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.278595 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.280196 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.291241 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1429/2867] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:22.296294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.306217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737b2859-6f73-4bd5-8496-d83cafd74276 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.311191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.311833 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.312114 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.312389 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] PATH_INFO: `/v3/projects/75fd199f475040fba28eb08bfde28ccb/users/32df42960d5d47eaaf1104ff3a6ccc14/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.327633 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fd199f475040fba28eb08bfde28ccb, user_id=32df42960d5d47eaaf1104ff3a6ccc14, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.330208 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.344282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1439/2868] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:22 2022] PUT /identity/v3/projects/75fd199f475040fba28eb08bfde28ccb/users/32df42960d5d47eaaf1104ff3a6ccc14/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:22.349626 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.360375 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46366e0e-4c8f-4c44-aac2-e557a80a304a admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.386284 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:22.434213 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.434943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.435106 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.435380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.436017 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.437608 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90416efd-161a-4bac-92c9-927ae01b7568 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.447341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1430/2869] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:22 2022] GET /identity/v3/roles => generated 1420 bytes in 99 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:22.451734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.462161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602e7de7-01ea-45b8-95ab-969f81779a77 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.467723 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.468371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.468651 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.468919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] PATH_INFO: `/v3/projects/75fd199f475040fba28eb08bfde28ccb/users/32df42960d5d47eaaf1104ff3a6ccc14/roles/6476749a9b1d4b73b7942cf1ab708d3c` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.470867 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=75fd199f475040fba28eb08bfde28ccb, user_id=32df42960d5d47eaaf1104ff3a6ccc14, role_id=6476749a9b1d4b73b7942cf1ab708d3c)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.473460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.484584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1440/2870] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:22 2022] PUT /identity/v3/projects/75fd199f475040fba28eb08bfde28ccb/users/32df42960d5d47eaaf1104ff3a6ccc14/roles/6476749a9b1d4b73b7942cf1ab708d3c => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:22.495693 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.495973 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.496244 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.523923 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] MFA Rules not processed for user `32df42960d5d47eaaf1104ff3a6ccc14`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:22.525127 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:22.596679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1431/2871] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:22 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:22.608078 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:22.609842 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:22.667594 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedb14dc-c665-4a5e-91f9-138f266cf2a8 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:22.671501 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:22.672103 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:22.672382 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:22.672652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:22.728105 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:22.729525 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:22.738669 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1441/2872] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:22 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:24.251130 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:24.252563 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:24.265554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af71cfb7-a68a-455e-92cf-896a2479d6eb None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:24.269090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:24.269662 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:24.270204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:24.270502 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:24.329156 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:24.330539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:24.339442 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1432/2873] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:24.854734 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:24.864287 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53d8c47-f76a-4dae-ad71-ab17a90f9152 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:24.867954 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:24.868547 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:24.868829 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:24.869096 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:24.879082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:24.880542 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d110ddef-5270-4537-8722-a2d4022e460a service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:24.890407 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1442/2874] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:24 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:28.770862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:28.809320 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=100 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:28.833956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8a60b-8059-4c30-9907-fbb59d9f9f8a service cinder] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:28.839024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:28.839653 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:28.839932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:28.840204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] PATH_INFO: `/v3/users/5fd372da71c643efbca21f7d18ffc4ac` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:28.842051 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5fd372da71c643efbca21f7d18ffc4ac)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:28.843989 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:28.889718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5fd372da71c643efbca21f7d18ffc4ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:28.896628 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5fd372da71c643efbca21f7d18ffc4ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:28.904494 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5fd372da71c643efbca21f7d18ffc4ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:28.905227 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5fd372da71c643efbca21f7d18ffc4ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:28.916768 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5fd372da71c643efbca21f7d18ffc4ac'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:28.922656 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1433/2875] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:28 2022] DELETE /identity/v3/users/5fd372da71c643efbca21f7d18ffc4ac => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:28.927158 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:28.927670 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:28.927789 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:28.946738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:28.947926 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:28.953439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:29.007371 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1443/2876] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:28 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:29.017596 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:29.019052 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:29.075289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b479008c-4c05-4fb8-9b11-4b010ac68d01 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:29.079547 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:29.080140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:29.080415 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:29.080686 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:29.089631 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:29.091138 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:29.100683 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1434/2877] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:29 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:29.392849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:29.394355 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:29.409403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fe9372-6fa4-46fa-8d93-452cf0cb63fc None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:29.414307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:29.414944 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:29.415226 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:29.415503 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] PATH_INFO: `/v3/projects/664c60dcf0b849bf86bde20095ab1ff3` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:29.416632 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] RBAC: Authorizing `identity:delete_project(project_id=664c60dcf0b849bf86bde20095ab1ff3)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:29.418581 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:29.468828 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '664c60dcf0b849bf86bde20095ab1ff3', 'request_id': 'req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.475017 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '664c60dcf0b849bf86bde20095ab1ff3', 'request_id': 'req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.482368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '664c60dcf0b849bf86bde20095ab1ff3', 'request_id': 'req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.494411 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] The token cache is being invalidate because project 664c60dcf0b849bf86bde20095ab1ff3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:29.494652 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.495444 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.497361 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1444/2878] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:29 2022] DELETE /identity/v3/projects/664c60dcf0b849bf86bde20095ab1ff3 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:29:29.501981 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:29.512338 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4800eb2f-df10-47c4-b01f-aedc07cf1931 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:29.567008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:29.573866 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:29.574718 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:29.575218 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:29.576359 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] PATH_INFO: `/v3/users/ce411d75aa164ab28ec6bc05be6bbd59` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:29.577782 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce411d75aa164ab28ec6bc05be6bbd59)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:29.580185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:29.626176 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce411d75aa164ab28ec6bc05be6bbd59'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.632988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce411d75aa164ab28ec6bc05be6bbd59'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.640316 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce411d75aa164ab28ec6bc05be6bbd59'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.640932 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce411d75aa164ab28ec6bc05be6bbd59'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.653193 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce411d75aa164ab28ec6bc05be6bbd59'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:29.659797 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1435/2879] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:29 2022] DELETE /identity/v3/users/ce411d75aa164ab28ec6bc05be6bbd59 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:29.938773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:29.940225 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:29.953849 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad52841-8f0a-4322-84e7-9dd6cf0f7749 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.011517 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.012052 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.012330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.012617 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] PATH_INFO: `/v3/projects/9376a9b34bbb46088035a49b97f20c40` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.014282 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9376a9b34bbb46088035a49b97f20c40)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.016167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:30.053552 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:30.064330 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9376a9b34bbb46088035a49b97f20c40', 'request_id': 'req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.070437 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9376a9b34bbb46088035a49b97f20c40', 'request_id': 'req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.078318 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9376a9b34bbb46088035a49b97f20c40', 'request_id': 'req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.088344 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] The token cache is being invalidate because project 9376a9b34bbb46088035a49b97f20c40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:30.088630 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.089246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.090738 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1445/2880] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:29 2022] DELETE /identity/v3/projects/9376a9b34bbb46088035a49b97f20c40 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:30.094825 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:30.096325 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:30.111301 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8e94378-8eae-443b-a84e-dc9528c5834e admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.174761 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.175395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.175678 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.175949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] PATH_INFO: `/v3/users/32df42960d5d47eaaf1104ff3a6ccc14` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.177080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] RBAC: Authorizing `identity:delete_user(user_id=32df42960d5d47eaaf1104ff3a6ccc14)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.179061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:30.224728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32df42960d5d47eaaf1104ff3a6ccc14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.232270 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32df42960d5d47eaaf1104ff3a6ccc14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.239579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32df42960d5d47eaaf1104ff3a6ccc14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.240192 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32df42960d5d47eaaf1104ff3a6ccc14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.251919 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:30.253108 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-6adc06b6-a108-4e18-ab84-fd123dddfcad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32df42960d5d47eaaf1104ff3a6ccc14'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.261925 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1436/2881] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:30 2022] DELETE /identity/v3/users/32df42960d5d47eaaf1104ff3a6ccc14 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Aug 30 04:29:30.544256 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:30.546421 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:30.561345 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3ba333-3bbc-4b0e-a5f1-b8bed2cafce5 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.619870 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.620490 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.620763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.621024 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] PATH_INFO: `/v3/projects/75fd199f475040fba28eb08bfde28ccb` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.622132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] RBAC: Authorizing `identity:delete_project(project_id=75fd199f475040fba28eb08bfde28ccb)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.624055 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:30.671163 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75fd199f475040fba28eb08bfde28ccb', 'request_id': 'req-a4d94d91-0a5f-4c7b-a706-dca66a441aef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.677602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75fd199f475040fba28eb08bfde28ccb', 'request_id': 'req-a4d94d91-0a5f-4c7b-a706-dca66a441aef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.686211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75fd199f475040fba28eb08bfde28ccb', 'request_id': 'req-a4d94d91-0a5f-4c7b-a706-dca66a441aef'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.697097 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] The token cache is being invalidate because project 75fd199f475040fba28eb08bfde28ccb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:30.697384 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.698013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a4d94d91-0a5f-4c7b-a706-dca66a441aef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:30.699505 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1446/2882] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:30 2022] DELETE /identity/v3/projects/75fd199f475040fba28eb08bfde28ccb => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:30.705744 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.706023 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.706358 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.727042 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:30.728257 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:30.789414 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1437/2883] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:30.798574 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.798850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.799112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.819051 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:30.820268 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:30.829554 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1447/2884] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:30 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:30.834102 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:30.845120 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8751568c-f731-487c-8f80-f1023390c9d0 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.849679 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.850356 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.850635 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.850904 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.851554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:30.851858 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.853439 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:30.865110 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1438/2885] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:30.869379 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:30.879292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd85888-1276-4d5f-b75e-b4c4ec60b972 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.883874 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.884460 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.884730 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.884990 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.885662 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.887589 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:30.915672 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1448/2886] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:30 2022] POST /identity/v3/projects => generated 373 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:30.919958 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:30.930097 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd98a70-7a06-43e5-a06d-e837a4218485 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:30.986639 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:30.987239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:30.987536 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:30.987786 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:30.988416 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:30.990342 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:31.011268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1439/2887] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:30 2022] POST /identity/v3/users => generated 287 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:31.015979 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:31.026625 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5780becb-a700-4fcd-8a13-f62e3b93b5e0 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:31.032031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:31.032636 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:31.032911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:31.033180 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:31.033814 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:31.035420 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:31.045336 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1449/2888] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:31.049146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:31.059059 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000a6061-8f66-442f-8398-81d4a7eab112 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:31.063783 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:31.064509 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:31.064634 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:31.064902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:31.065541 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:31.067166 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48634e29-c5f3-4edb-b266-75ec8c0bfe34 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:31.077736 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1440/2889] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:31.081732 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:31.092773 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b80bf5-fc3f-4ec5-9603-f9d9d42d450e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:31.097286 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:31.097923 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:31.098194 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:31.098489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] PATH_INFO: `/v3/projects/8652672950004ca0bd9912537a583b72/users/bd0f822076f54525a4486d3a2ea1c727/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:31.110375 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:31.114524 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] RBAC: Authorizing `identity:create_grant(project_id=8652672950004ca0bd9912537a583b72, user_id=bd0f822076f54525a4486d3a2ea1c727, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:31.117905 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:31.131159 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1450/2890] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:31 2022] PUT /identity/v3/projects/8652672950004ca0bd9912537a583b72/users/bd0f822076f54525a4486d3a2ea1c727/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:31.143915 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3d8a-7520-42d9-b493-b968ea98399d None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:31.144449 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3d8a-7520-42d9-b493-b968ea98399d None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:31.144759 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32f3d8a-7520-42d9-b493-b968ea98399d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:31.176765 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-a32f3d8a-7520-42d9-b493-b968ea98399d None None] MFA Rules not processed for user `bd0f822076f54525a4486d3a2ea1c727`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:31.178004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-a32f3d8a-7520-42d9-b493-b968ea98399d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:31.226855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:31.244657 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1441/2891] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:31.256378 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:31.258074 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:31.316231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23677b80-40ad-40fa-b73e-493ac1bf0c1f admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:31.319718 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:31.320312 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:31.320584 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:31.320848 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:31.329557 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:31.330989 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:31.339495 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1451/2892] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:32.893801 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:32.894383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:32.894383 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:32.913585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] MFA Rules not processed for user `bd0f822076f54525a4486d3a2ea1c727`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:32.914832 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:32.922092 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1442/2893] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:33.604049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:33.605864 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:33.684986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c8f8ee-aeaf-47f8-952c-4ac78a866d69 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:33.690851 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:33.691883 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:33.692198 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:33.692527 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:33.694671 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:33.709232 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:33.711252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:33.723912 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1452/2894] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:33 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:35.645851 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:35.647432 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:35.710956 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3b227b-c5e7-4a50-ac7b-9951846e1063 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:35.714804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:35.715480 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:35.715747 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:35.716112 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:35.718091 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:35.727987 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:35.729411 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd3a69d-bf00-4df3-85ee-e2a18c0f645a service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:35.737821 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1443/2895] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:35 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:43.719084 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=61,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:45.734135 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=33 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:46.584590 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:46.586380 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:46.689583 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3116feb7-6579-4168-936b-dc08c1cf1624 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:46.695300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:46.695972 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:46.696252 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:46.696520 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:46.698394 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:46.717638 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:46.722369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:46.734610 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1453/2896] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:46 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:47.307619 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d3cb-f672-47aa-8a18-d5ea22e2aaac None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:47.308089 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d3cb-f672-47aa-8a18-d5ea22e2aaac None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:47.308194 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb68d3cb-f672-47aa-8a18-d5ea22e2aaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:47.367985 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-eb68d3cb-f672-47aa-8a18-d5ea22e2aaac None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:47.369205 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-eb68d3cb-f672-47aa-8a18-d5ea22e2aaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:47.433620 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1444/2897] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:29:47 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:47.480891 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:47.482682 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:47.572168 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9421609-ebb1-4afc-9b48-e27e06191fe4 service nova] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:47.578069 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:47.578924 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:47.579064 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:47.579250 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:47.587484 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:47.589044 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:47.598498 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1454/2898] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:29:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:29:48.101962 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd936fb-30c1-475b-9db8-6818d3643103 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.102263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd936fb-30c1-475b-9db8-6818d3643103 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.102544 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd936fb-30c1-475b-9db8-6818d3643103 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.121428 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-dcd936fb-30c1-475b-9db8-6818d3643103 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:48.122687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-dcd936fb-30c1-475b-9db8-6818d3643103 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:48.129888 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1445/2899] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:29:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:48.143112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:48.155241 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8935b7f-7319-4d3b-9722-07b8799aec86 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:48.161553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:48.162385 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.162689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.162958 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.173622 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:48.175302 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:48.183995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1455/2900] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:29:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:29:48.399066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262558c-f7fd-4256-bc6a-9867225a9337 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.399542 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262558c-f7fd-4256-bc6a-9867225a9337 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.399648 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6262558c-f7fd-4256-bc6a-9867225a9337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.419409 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-6262558c-f7fd-4256-bc6a-9867225a9337 None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:48.420723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-6262558c-f7fd-4256-bc6a-9867225a9337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:48.428200 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1446/2901] 10.222.0.46 () {58 vars in 1009 bytes} [Tue Aug 30 04:29:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:48.441627 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:48.456272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08d1af1-641e-4e5d-bce5-958447e4dde8 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:48.462500 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:48.463111 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.463391 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.463664 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.472886 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:48.474489 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:48.482663 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1456/2902] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:29:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:29:48.813556 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.813868 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.814116 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.833912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] MFA Rules not processed for user `f3da3928b5414f14ae49e96282a135f1`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:48.835204 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:48.842827 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1447/2903] 10.222.0.46 () {58 vars in 967 bytes} [Tue Aug 30 04:29:48 2022] POST /identity/v3/auth/tokens => generated 3195 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:48.855031 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:48.866424 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9d972e6-5997-4a85-8cd6-9b5b95244af5 service swift] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:48.871368 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'fb30d87ff47b49e39091ef401f10ff59', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:48.871952 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:48.872764 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:48.873034 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:48.880951 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:48.882514 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f946410e-ae3d-4f5c-befd-291b2f02a88c service swift] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:48.890689 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1457/2904] 10.222.0.46 () {60 vars in 1384 bytes} [Tue Aug 30 04:29:48 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Aug 30 04:29:50.993137 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.056949 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d4a417-9b5d-427d-a910-c19dddb4a49c None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.061814 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.062486 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.062766 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.063066 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] PATH_INFO: `/v3/users/bd0f822076f54525a4486d3a2ea1c727` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.064268 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bd0f822076f54525a4486d3a2ea1c727)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.066365 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.114892 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:51.116516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bd0f822076f54525a4486d3a2ea1c727'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.124525 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bd0f822076f54525a4486d3a2ea1c727'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.131990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bd0f822076f54525a4486d3a2ea1c727'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.132687 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bd0f822076f54525a4486d3a2ea1c727'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.145520 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bd0f822076f54525a4486d3a2ea1c727'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.151394 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1448/2905] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:29:50 2022] DELETE /identity/v3/users/bd0f822076f54525a4486d3a2ea1c727 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:51.156508 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.157005 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.157118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.176370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:51.177591 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:51.187107 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=100 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:51.248041 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1458/2906] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:51.257703 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.259170 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:51.318579 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53d790e-3653-4f70-a195-8942709bd084 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.322210 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.322840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.323119 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.323390 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.333728 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.335208 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03027879-3a51-4866-839a-d18031bcfdaa service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.345707 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1449/2907] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:51 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:51.521275 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.522911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:51.548307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ca7b386-8ecd-451f-a0ad-c97170ed6394 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.553911 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.554702 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.555094 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.555475 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] PATH_INFO: `/v3/projects/8652672950004ca0bd9912537a583b72` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.556767 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8652672950004ca0bd9912537a583b72)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.558978 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.617079 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8652672950004ca0bd9912537a583b72', 'request_id': 'req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.623799 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8652672950004ca0bd9912537a583b72', 'request_id': 'req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.632184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8652672950004ca0bd9912537a583b72', 'request_id': 'req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.643417 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] The token cache is being invalidate because project 8652672950004ca0bd9912537a583b72 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81502) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:29:51.643728 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.644403 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-34da8bea-d75c-49da-bf6a-7d62a5e0caa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:29:51.646222 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1459/2908] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:29:51 2022] DELETE /identity/v3/projects/8652672950004ca0bd9912537a583b72 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:51.651309 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.651587 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.651855 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.673455 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:51.674944 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:51.743566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1450/2909] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:51.752879 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.753235 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.753580 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.777061 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:51.778439 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:51.786932 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1460/2910] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:51 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Aug 30 04:29:51.790984 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.801751 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454790eb-cdd5-4a51-b6be-75fc4eff013e None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.806539 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.807197 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.807508 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.807772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] PATH_INFO: `/v3/domains` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.808461 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=81503) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Aug 30 04:29:51.808772 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.811237 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.823553 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1451/2911] 10.222.0.46 () {58 vars in 1153 bytes} [Tue Aug 30 04:29:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Aug 30 04:29:51.828086 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.837108 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f412b4a8-a3ff-4168-a03e-83d016b13e0f None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.842164 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.842788 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.843060 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.843317 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] PATH_INFO: `/v3/projects` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.843940 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.845784 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.873960 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1461/2912] 10.222.0.46 () {60 vars in 1153 bytes} [Tue Aug 30 04:29:51 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:51.878310 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.888289 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f08e2ca-dc24-4f78-92ea-d6a21738c930 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.936731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:51.949125 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.949784 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.950065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.950762 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] PATH_INFO: `/v3/users` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.951401 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.953315 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:51.976974 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1452/2913] 10.222.0.46 () {60 vars in 1144 bytes} [Tue Aug 30 04:29:51 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Aug 30 04:29:51.978205 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:51.989697 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9206891-581b-4f73-bc8a-0e8e15f124dc admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:51.995018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:51.995546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:51.995766 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:51.996049 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:51.996704 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:51.999137 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:52.010118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1462/2914] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:51 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Aug 30 04:29:52.014140 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:52.024395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d32b6-6ed2-44bc-9df2-49479e665257 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:52.029160 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:52.029694 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:52.029969 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:52.030239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] PATH_INFO: `/v3/roles` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:52.030912 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:52.032511 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7fc55d3-0e7d-4e28-aa7d-c447e208d4db admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:52.043189 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1453/2915] 10.222.0.46 () {58 vars in 1122 bytes} [Tue Aug 30 04:29:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Aug 30 04:29:52.048070 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:52.058057 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1dcdfb0-5be5-45eb-a6c5-8df45825c0d2 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:52.063346 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:52.063986 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] REQUEST_METHOD: `PUT` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:52.064262 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:52.064530 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] PATH_INFO: `/v3/projects/3b90a52b66304d6a8396c2989bcc612b/users/e76baa3c1f8d48aca5c5109327e28640/roles/65feaeff5c0046dbaf41270fb80ff8cd` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:52.079474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b90a52b66304d6a8396c2989bcc612b, user_id=e76baa3c1f8d48aca5c5109327e28640, role_id=65feaeff5c0046dbaf41270fb80ff8cd)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:52.082228 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:52.095369 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1463/2916] 10.222.0.46 () {60 vars in 1483 bytes} [Tue Aug 30 04:29:52 2022] PUT /identity/v3/projects/3b90a52b66304d6a8396c2989bcc612b/users/e76baa3c1f8d48aca5c5109327e28640/roles/65feaeff5c0046dbaf41270fb80ff8cd => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:29:52.100465 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b3f-8ad4-4f4a-beb7-5324808a6401 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:52.100748 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b3f-8ad4-4f4a-beb7-5324808a6401 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:52.101041 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a04b3f-8ad4-4f4a-beb7-5324808a6401 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:52.120943 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-70a04b3f-8ad4-4f4a-beb7-5324808a6401 None None] MFA Rules not processed for user `01e05990467f46b9b8ba90e062973118`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:52.122222 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-70a04b3f-8ad4-4f4a-beb7-5324808a6401 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:52.181555 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1454/2917] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:52 2022] POST /identity/v3/auth/tokens => generated 3239 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:52.191585 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:52.193298 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:52.233013 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:52.254217 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac645f1-ea92-466a-8a70-2dc4fa5c3410 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:52.258000 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:52.258637 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:52.258914 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:52.259181 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:52.268969 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:52.270649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:52.279995 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1464/2918] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:52 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Aug 30 04:29:52.967004 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0dde-e86d-478a-ab47-10c6d3734e55 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:52.967303 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0dde-e86d-478a-ab47-10c6d3734e55 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:52.967585 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cff0dde-e86d-478a-ab47-10c6d3734e55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:53.002902 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-4cff0dde-e86d-478a-ab47-10c6d3734e55 None None] MFA Rules not processed for user `e76baa3c1f8d48aca5c5109327e28640`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:53.004185 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4cff0dde-e86d-478a-ab47-10c6d3734e55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:53.076201 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1455/2919] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:52 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:53.087818 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:53.089291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:53.150370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c10719d-e1b7-4b67-acbe-a2522b3ec599 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:53.154890 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cefcf250ee17414389d3014e8755416e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:53.155492 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:53.155768 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:53.156062 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:53.164862 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:53.166311 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:53.177045 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1465/2920] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:53 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:54.573804 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14293-2cdc-4e9e-ab4b-38b0df75d942 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:54.574380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14293-2cdc-4e9e-ab4b-38b0df75d942 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:54.574380 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f14293-2cdc-4e9e-ab4b-38b0df75d942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:54.595576 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-35f14293-2cdc-4e9e-ab4b-38b0df75d942 None None] MFA Rules not processed for user `e76baa3c1f8d48aca5c5109327e28640`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:54.596819 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-35f14293-2cdc-4e9e-ab4b-38b0df75d942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:54.604516 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1456/2921] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:54 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:29:54.883419 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:29:54.884871 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:54.898546 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb4ebdf5-32ba-4489-bd74-852849a7d2d3 service cinder] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:29:54.902291 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:29:54.902894 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:54.903177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:54.904012 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:54.905919 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:54.916518 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:29:54.918184 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:29:54.926973 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1466/2922] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:29:54 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:29:55.188724 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] REQUEST_METHOD: `POST` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:29:55.189008 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:29:55.189275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:29:55.210674 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.auth.core [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] MFA Rules not processed for user `b301d6b1d85b4371abde52727d9b387e`. Rule list: `[]` (Enabled: `True`). {{(pid=81503) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:29:55.211873 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:29:55.272328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:29:55.282058 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1457/2923] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:29:55 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:30:04.244167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:30:04.246370 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:30:04.353491 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-991a7233-8847-4c40-985f-615316841d55 service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:30:04.360603 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:30:04.361213 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:30:04.361494 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:30:04.361763 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:30:04.363553 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:30:04.376676 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:30:04.378655 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e309a01f-d994-4a6d-b438-b9463e338a86 service nova] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:30:04.390395 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1467/2924] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:30:04 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:30:14.384962 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=76 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:30:14.642598 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:30:14.644061 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:30:14.662816 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d294ca9a-9fd6-4e3d-98aa-64846ebd78a6 None None] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:30:14.668713 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '148e575ef9b642c8b81d7eca8dd70bfa', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service', 'ResellerAdmin', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:30:14.669341 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:30:14.669606 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:30:14.669875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:30:14.672492 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:30:14.685251 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:30:14.686952 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:30:14.700820 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1458/2925] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:30:14 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:30:24.694376 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:30:58.424754 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:30:58.425294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:30:58.425294 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:30:58.445022 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] MFA Rules not processed for user `e76baa3c1f8d48aca5c5109327e28640`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:30:58.446272 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:30:58.453075 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1468/2926] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:30:58 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:31:08.438624 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=3 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:31:53.947992 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:31:53.949372 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:31:54.011629 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:31:54.025231 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5cd13cb-a273-4d54-abf8-79a29c4c1a2d None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:31:54.030497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:31:54.031210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:31:54.031497 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:31:54.031769 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] PATH_INFO: `/v3/users/e76baa3c1f8d48aca5c5109327e28640` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:31:54.032892 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e76baa3c1f8d48aca5c5109327e28640)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:31:54.033322 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8a578d-01e5-4b51-afdb-a7fd7bb6109c service nova] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:31:54.034831 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:31:54.039101 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:31:54.039712 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:31:54.039990 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:31:54.040263 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:31:54.042030 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:31:54.054836 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:31:54.056426 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:31:54.068554 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1459/2927] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:31:53 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:31:54.094348 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e76baa3c1f8d48aca5c5109327e28640'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.107007 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e76baa3c1f8d48aca5c5109327e28640'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.107779 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e76baa3c1f8d48aca5c5109327e28640'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.108414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e76baa3c1f8d48aca5c5109327e28640'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.116177 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-a43b1f90-a1c3-4551-8109-3326009c575c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e76baa3c1f8d48aca5c5109327e28640'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.129499 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1469/2928] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:31:54 2022] DELETE /identity/v3/users/e76baa3c1f8d48aca5c5109327e28640 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:31:54.303090 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:31:54.313875 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722b5de8-2371-43b5-a75b-6fe4273edd56 service placement] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:31:54.383862 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:31:54.384484 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:31:54.384757 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:31:54.385024 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] PATH_INFO: `/v3/projects/3b90a52b66304d6a8396c2989bcc612b` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:31:54.386126 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b90a52b66304d6a8396c2989bcc612b)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:31:54.388080 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:31:54.434065 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b90a52b66304d6a8396c2989bcc612b', 'request_id': 'req-cdf5c47f-a078-47a8-a296-bd05672d4860'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.442795 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b90a52b66304d6a8396c2989bcc612b', 'request_id': 'req-cdf5c47f-a078-47a8-a296-bd05672d4860'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.449239 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b90a52b66304d6a8396c2989bcc612b', 'request_id': 'req-cdf5c47f-a078-47a8-a296-bd05672d4860'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.459436 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] The token cache is being invalidate because project 3b90a52b66304d6a8396c2989bcc612b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:31:54.459723 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.460354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:31:54.461840 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1460/2929] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:31:54 2022] DELETE /identity/v3/projects/3b90a52b66304d6a8396c2989bcc612b => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:32:00.443260 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] REQUEST_METHOD: `POST` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:32:00.443792 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:32:00.443908 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:32:00.463118 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.auth.core [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] MFA Rules not processed for user `b301d6b1d85b4371abde52727d9b387e`. Rule list: `[]` (Enabled: `True`). {{(pid=81502) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Aug 30 04:32:00.464307 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:32:00.521606 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1470/2930] 10.222.0.46 () {58 vars in 958 bytes} [Tue Aug 30 04:32:00 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Aug 30 04:32:01.049487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:32:01.051844 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:32:01.074292 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:32:01.111566 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf5c47f-a078-47a8-a296-bd05672d4860 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:32:01.115771 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:32:01.116373 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:32:01.116650 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:32:01.116918 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:32:01.118669 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:32:01.128471 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:32:01.129988 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:32:01.138180 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1461/2931] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:32:01 2022] GET /identity/v3/auth/tokens => generated 3121 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:32:07.406795 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:32:07.408167 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:32:07.490004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d736195a-56dc-4b46-befa-c38dea53ba75 None None] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:32:07.495039 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:32:07.495649 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:32:07.495925 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:32:07.496191 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:32:07.497933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:32:07.509948 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:32:07.511541 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:32:07.523148 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1471/2932] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:32:07 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:32:11.134514 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=17 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:32:17.517466 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=1 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:33:45.001294 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:33:45.010637 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbb3694-d628-4690-9986-04c7d9e5cdb5 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:33:45.014518 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:33:45.015354 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:33:45.015624 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:33:45.015886 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:33:45.017593 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:33:45.028788 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:33:45.030187 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:33:45.039099 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1462/2933] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:33:45 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:33:54.037333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:33:54.047631 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fab96f-706b-4738-b3c5-6e0ace3ad203 service placement] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:33:54.051211 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:33:54.051804 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:33:54.052082 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:33:54.052351 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:33:54.054740 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:33:54.127577 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:33:54.129004 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:33:54.140961 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1472/2934] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:33:54 2022] GET /identity/v3/auth/tokens => generated 3372 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:33:55.034842 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=6 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:33:58.527946 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:33:58.529395 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:33:58.540789 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f2ce123-6f16-42bc-878a-501196a9b84e service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:33:58.544564 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '5c3ad8bf2cc8457eb6685a514e545dbd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:33:58.545157 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] REQUEST_METHOD: `GET` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:33:58.545435 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:33:58.545705 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:33:58.547599 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:33:58.611887 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:33:58.613328 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:33:58.623431 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1463/2935] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:33:58 2022] GET /identity/v3/auth/tokens => generated 3239 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:34:04.135612 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=34 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:34:04.785246 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:34:04.790364 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:34:04.810132 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08475f52-7e64-40cd-b1da-e35aafc6ae0d service neutron] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:34:04.815645 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:34:04.816362 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:34:04.817243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:34:04.817243 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] PATH_INFO: `/v3/users/b301d6b1d85b4371abde52727d9b387e` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:34:04.818372 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b301d6b1d85b4371abde52727d9b387e)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:34:04.820300 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:34:04.869981 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b301d6b1d85b4371abde52727d9b387e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:04.881751 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b301d6b1d85b4371abde52727d9b387e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:04.882478 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b301d6b1d85b4371abde52727d9b387e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:04.883112 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b301d6b1d85b4371abde52727d9b387e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:04.891794 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b301d6b1d85b4371abde52727d9b387e'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:04.906910 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1473/2936] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:34:04 2022] DELETE /identity/v3/users/b301d6b1d85b4371abde52727d9b387e => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:34:05.085550 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:34:05.096259 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e538d9ed-2372-47a7-93fa-7eea81132347 service neutron] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:34:05.155330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:34:05.155959 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:34:05.156238 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:34:05.156512 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] PATH_INFO: `/v3/projects/59c345f38d6548cd820affce5406d62c` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:34:05.157625 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59c345f38d6548cd820affce5406d62c)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:34:05.159809 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:34:05.206731 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59c345f38d6548cd820affce5406d62c', 'request_id': 'req-2b54da27-f266-4d81-81b8-d6bf9a690877'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.214798 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59c345f38d6548cd820affce5406d62c', 'request_id': 'req-2b54da27-f266-4d81-81b8-d6bf9a690877'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.221330 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59c345f38d6548cd820affce5406d62c', 'request_id': 'req-2b54da27-f266-4d81-81b8-d6bf9a690877'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.231577 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] The token cache is being invalidate because project 59c345f38d6548cd820affce5406d62c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:34:05.231878 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.232472 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.233942 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1464/2937] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:34:05 2022] DELETE /identity/v3/projects/59c345f38d6548cd820affce5406d62c => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:34:05.237933 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:34:05.239399 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:34:05.252534 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315737f4-50e2-484d-9f13-3733b51ca51e admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:34:05.308850 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:34:05.309474 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:34:05.309749 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:34:05.310020 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] PATH_INFO: `/v3/users/97bf0511074e409a8b73b615aef5971b` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:34:05.311331 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97bf0511074e409a8b73b615aef5971b)` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:34:05.313292 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:34:05.360219 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97bf0511074e409a8b73b615aef5971b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.371941 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97bf0511074e409a8b73b615aef5971b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.372602 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97bf0511074e409a8b73b615aef5971b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.373200 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97bf0511074e409a8b73b615aef5971b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.380414 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.notifications [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97bf0511074e409a8b73b615aef5971b'} {{(pid=81502) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.382715 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:34:05.395595 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1474/2938] 10.222.0.46 () {58 vars in 1224 bytes} [Tue Aug 30 04:34:05 2022] DELETE /identity/v3/users/97bf0511074e409a8b73b615aef5971b => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:34:05.564815 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Authenticating user token {{(pid=81503) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:34:05.566275 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.fernet_utils [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81503) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:34:05.579870 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b54da27-f266-4d81-81b8-d6bf9a690877 admin admin] Validating token access rules against request {{(pid=81503) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:34:05.600725 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:34:05.638266 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '01e05990467f46b9b8ba90e062973118', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'b409aaa1029446de8dc3bfb718ab67c6', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81503) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:34:05.638965 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] REQUEST_METHOD: `DELETE` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:34:05.639487 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] SCRIPT_NAME: `/identity` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:34:05.639743 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] PATH_INFO: `/v3/projects/dbd4de50617f4a3585cb620eb58a73c2` {{(pid=81503) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:34:05.640853 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd4de50617f4a3585cb620eb58a73c2)` {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:34:05.642869 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] RBAC: Authorization granted {{(pid=81503) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:34:05.690150 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd4de50617f4a3585cb620eb58a73c2', 'request_id': 'req-24996884-b7a8-4758-a4b1-b7da0a88412d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.698148 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd4de50617f4a3585cb620eb58a73c2', 'request_id': 'req-24996884-b7a8-4758-a4b1-b7da0a88412d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.704673 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd4de50617f4a3585cb620eb58a73c2', 'request_id': 'req-24996884-b7a8-4758-a4b1-b7da0a88412d'} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.715146 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] The token cache is being invalidate because project dbd4de50617f4a3585cb620eb58a73c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=81503) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Aug 30 04:34:05.715430 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.716057 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG keystone.notifications [None req-24996884-b7a8-4758-a4b1-b7da0a88412d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=81503) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Aug 30 04:34:05.717535 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: [pid: 81503|app: 0|req: 1465/2939] 10.222.0.46 () {58 vars in 1233 bytes} [Tue Aug 30 04:34:05 2022] DELETE /identity/v3/projects/dbd4de50617f4a3585cb620eb58a73c2 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Aug 30 04:34:15.394869 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=4 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:34:15.712229 dv-cinder-852208-11-iscsi devstack@keystone.service[81503]: DEBUG dbcounter [-] [81503] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=81503) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Aug 30 04:37:10.520018 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Authenticating user token {{(pid=81502) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Aug 30 04:37:10.521412 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:37:10.665210 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac533b-46ba-475a-8135-43fd8b6008a7 admin admin] Validating token access rules against request {{(pid=81502) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Aug 30 04:37:10.723406 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ba39d57469d34f258aa903bff3935c3f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '13e34a5a11cc4678908121bcb19d9359', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=81502) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Aug 30 04:37:10.724008 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] REQUEST_METHOD: `GET` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Aug 30 04:37:10.724279 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] SCRIPT_NAME: `/identity` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Aug 30 04:37:10.724544 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=81502) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Aug 30 04:37:10.726333 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.fernet_utils [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=81502) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Aug 30 04:37:10.737449 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Aug 30 04:37:10.739076 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c3c160f-e434-4a16-a236-7143d2cd0abe service placement] RBAC: Authorization granted {{(pid=81502) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Aug 30 04:37:10.749401 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: [pid: 81502|app: 0|req: 1475/2940] 10.222.0.46 () {60 vars in 1365 bytes} [Tue Aug 30 04:37:10 2022] GET /identity/v3/auth/tokens => generated 3308 bytes in 230 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Aug 30 04:37:20.745161 dv-cinder-852208-11-iscsi devstack@keystone.service[81502]: DEBUG dbcounter [-] [81502] Writing DB stats keystone:SELECT=89 {{(pid=81502) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}